Methods and systems for displaying text using RSVP

ABSTRACT

The disclosure describes systems and methods related to moving object content that is enhanced with textual content presented using rapid serial visual presentation (“RSVP”). In one embodiment, video content is enhanced with RSPV content. Related embodiments provide systems and methods for displaying, creating, and editing such content. A particular embodiment provides for RSPV enhanced video previews. In another embodiment, augmented reality content is enhanced with RSVP content. In some embodiments, RSPV enhanced moving object content is provided on various end user electronic devices including smart phones, smart watches, and head mounted displays. These and other embodiments are disclosed further herein.

CROSS REFERENCE TO RELATED APPLICATIONS

This application is a continuation-in-part of U.S. application Ser. No.13/730,163 filed on Dec. 28, 2012. This application is also related toU.S. application Ser. No. 13/547,982 filed on Jul. 12, 2012. The entiredisclosures of both of these applications are hereby incorporated byreference.

BACKGROUND

The present invention relates to apparatuses and methods for enablingimproved display of textual content on an electronic display.

Communications devices, including PCs, smartphones, tablets, e-readers,etc. continue to grow in popularity, and have become an integral part ofboth personal and business communications. As a result, users continueto spend more time using their communications devices during the courseof a day reading e-mails, reading web sites, sending short messageservice (SMS) messages, etc.

Rapid serial visual presentation (RSVP) is a method of displayingtextual content in which each word of the textual content is displayedin sequential order, one at a time, at a certain display rate, at afixed location on a display. RSVP was first introduced in the 1970s as atechnique for presenting text one word at a time in a display. Manyreferences since then have provided information on the use of RSVP in avariety of applications. Commercially available products based on RSVPinclude “Zap Reader” (www.zapreader.com/reader) and “Spreeder”(www.spreeder.com). Some prior methods exist for improving theeffectiveness of an RSVP by varying the display time of a word in thedisplay based on word length and word type (see, U.S. Pat. No. 6,130,968to McIan et al. (“McIan”)) and based on word frequency (see WO/37256 byGoldstein et al. (“Goldstein 2002”)). While these techniques arebeneficial in improving comprehension of the displayed text, newtechniques and methods are needed to further increase a user's readingspeed, and improve the presentation of dense content on electronicdisplays.

Isolated efforts have also been made to apply RSVP to particularapplications (e.g., email applications) in mobile communications devices(see, US 2011/0115819 to Hanson). However, the challenges andopportunities for integrating RSVP into user interfaces for increasingthe density of displayable content remain largely unexplored.

SUMMARY

Despite heavy technological (digital) advances, the illustration oftextual information on electronic displays has not fundamentallychanged. Textual information is typically displayed in lines such thatthe reader's eye moves sequentially from word to word. However,interacting with text displayed in the traditional manner on acommunications device, and particularly a mobile communications device,presents certain inconveniences to a user. For example, the display areaof a mobile communications device may be limited, which may increase thetime spent reading an e-mail or a web site, as the user may have toscroll through multiple pages to read the entire e-mail or web site.

Previous implementations of Rapid Serial Visual Presentation (RSVP) donot address using RSVP to improve user access to information from ahomescreen (e.g., a “homescreen” of a smartphone running a mobileoperating system such as iOS™, Android™, or Windows Phone™ or a“desktop” screen of a PC, laptop, etc., running an operating system suchas Windows™, or Mac™ OS; or a homescreen/desktop screen of anintermediate portable device such as notepad, touchpad, etc running acorresponding operating system; all referenced herein as simply a“homescreen” for simplicity) user interface or how to best integrateRSVP into a variety of applications. Given the increasing reliance onsmall-screen devices (particularly mobile communication devices, butalso notebook computers and other highly portable computing devices) fora variety of purposes, there is a growing need to efficiently utilizescreen space within user interfaces. Moreover, even with respect tolarger screen devices, there are unexplored opportunities to create moreefficient interfaces by incorporating RSVP techniques. Embodiments ofthe invention relate to electronic interfaces that effectively utilizeRSVP to improve user access to information.

In one embodiment, a communications device displays a first iconrepresenting a notification event associated with an application bydisplaying the first icon with an icon representing the application. Thefirst icon further represents a presence of content that is displayableusing RSVP. The communications device receives a first user interfaceaction to select the notification event, and in response to the firstuser interface action, displays textual content associated with thenotification event in a designated display area using RSVP. The RSVPcontent may contain embedded text, a uniform resource locator (URL), oran attachment. If the user selects embedded text, the RSVP contentcorresponding to the embedded text may be displayed in the designateddisplay area using RSVP. In one embodiment, if the user selects a URL,the contents of the webpage corresponding to the URL may be displayed inthe designated display area using RSVP. Alternatively, the webpage maybe displayed by a browser application. If the user selects anattachment, an application associated with the attachment may belaunched to open the attachment. For example, if the attachment is aphoto, a photo viewer application may be launched to open the photo.

In another embodiment, the communications device displays a first iconwithin an application interface. The first icon represents presence oftextual content that is displayable using RSVP. The communicationsdevice receives a first user interface action to select the first icon,and in response to the first user interface action, displays the textualcontent in a designated display area using RSVP. In an alternative tothis embodiment, an application interface is configurable so that anyselectable item within the application interface that is associated withtextual content may have that content displayed using RSVP when the itemis selected.

In yet another embodiment, a search query is received by at least oneserver computer. At least one search result corresponding to the searchquery is transmitted to a browser application for display in the browserapplication on a user device. The search result is configured to bedisplayed by the browser application with an icon representing presenceof content, corresponding to the at least one search result, that isdisplayable using RSVP.

In yet another embodiment, RSVP content may be embedded in a map, photo,diagram, video, presentation, etc. A map, photo, diagram, video, orpresentation may be displayed by the appropriate application. A user mayspecify whether to add “global” RSVP content and/or “local” RSVPcontent. If the user chooses to add “global” (e.g., in reference to adocument in its entirety) RSVP content, an interface which allows theuser to add the “global” RSVP content may be displayed. If the userchooses to add “local” RSVP content, an interface which allows the userto specify location(s), element(s), and/or text selection, and add thecorresponding “local” RSVP content to be displayed. For example, theuser may specify locations on photos, maps, etc., or specify elements orobjects in photos, diagrams, videos and presentations, etc. In oneembodiment, the interface may also allow selection of text such thatRSVP content may be associated with the selected text. In the case of aphoto, in addition to specifying locations on the photo, the user mayselect areas of the photo, such as, for example a face of a person.

There is also a need to minimize the need to download or stream largeamounts of content to a wireless device, especially prior to a usermaking a consumption decision regarding the content. Multimedia contentincluding video and/or audio is an example of a type of content thatcreates such issues. This is particularly a problem when accessing suchfiles on wireless devices, which often have more limited bandwidthand/or lower imposed data limits on services than do devices with ahardline connection. Therefore, another embodiment of the presentinvention provides an interface for receiving information about videoand other multi-media content via RSVP display without having todownload and/or stream the content. In a particular embodiment, a videoitem in a list of video items is selectable to initiate display ofinformation via RSVP regarding that video item. In a particularembodiment, at least a portion of the list of video items remaineddisplayed while information is displayed via RSPV regarding the selecteditem.

There is also often a need to provide textual content that is associatedwith video, particularly given the increasingly interactive natureonline video sharing. However, existing methods of providing textualcontent associated with video include a first method in which text isprovided entirely outside of the video display window (e.g., in commentsareas below the video window) and a second method in which text isprovided within the video interface using traditional line by line textpresentation (e.g., subtitles). Both of these methods havedisadvantages. In the first method, because the text is provide outsidethe video window, the user must look away from the video content to viewthe text. Also, this method generally is not convenient for associatingtext with a particular time portion of the video content. In the secondmethod, because the text is presented for traditional left-to-right (orright-to-left) reading, it necessarily obscures a significant portion ofthe video window. Also, because the traditional method of textpresentation does not allow for the fastest reading speeds with a highlevel of comprehension, consuming text by traditional methods during avideo is more disruptive of video content than would be necessary usingthe improved methods of certain embodiments of the present invention.

Therefore, in another embodiment, RSVP content is embedded in videocontent for display at selected points during the video. A particularembodiment provides one or more icons (i.e., any form of visualindicator) along a video timeline bar that can be selected to view textvia RSVP that is associated with that portion of the video. Anotherparticular embodiment displays one or more icons within the videocontent that are selectable to view RSVP content associated with thatportion of the video content. In another particular embodiment, one ormore icons are placed on and track one or more recognized objects withinthe video content, the one or more icons being selectable to view RSVPcontent associated with the corresponding video object. In one versionof this embodiment, the icon remains displayed on a moving video objectwhile the object remains displayed in the video. Some embodiments areconfigured such that RSVP content embedded in or otherwise associatedwith a video file displays automatically at particular times duringvideo playback without the need for user selection, or such thatautomatic RSVP content display is a default or non-default selectablesetting that may be altered via a user settings interface. In anotherembodiment, a preview version of a video is provided that displays shortclips of different portions of the video after (or, alternatively,either before or while) displaying RSVP content associated with thoseclips to provide efficient, low-data video previews while enhancing theinformation provided to the user. A related embodiment is provided foradding RSVP content to videos and for creating video previews thatinclude RSVP content.

In one embodiment of the invention, RSVP content is provided within aninterface of an augmented reality application. A particular embodimentdisplays one or more icons on or near one or more real world objectsviewed through the augmented reality application, the one or more iconsbeing selectable to view RSVP content associated the corresponding realworld object.

Some embodiments of the invention are implemented in or with particulartypes of user devices such as, for example, desktop computers andportable electronic devices such as mobile phones, smartphones,multi-media players, e-readers, tablet/touchpad, notebook or laptop PCs,smart watches, smart televisions, and other communication devices. Someembodiments of the invention include implementation in particular typesof electronic devices including, for example, smart glasses and othertypes of head mounted displays.

These and other embodiments are more fully described below.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a diagram of a computer system in the context of a computernetwork in accordance with one embodiment of the present invention.

FIG. 2a is a block diagram of exemplary software modules included in thecomputer program product loaded on the user device of FIG. 1 inaccordance with one embodiment of the present invention.

FIG. 2b shows a specific implementation of the embodiment of FIG. 2a inwhich the specific RSVP-enabled applications are video-relatedapplications.

FIG. 2c shows a specific implementation of the embodiment of FIG. 2aincluding an RSVP-enabled augmented reality (AR) application.

FIG. 2d shows server-side architecture for a server of FIG. 1 configuredto implement server-side components of an RVSP-enabled augmented reality(AR) service.

FIG. 3 is a block diagram of exemplary software modules included in theRSVP library of FIG. 2a in accordance with one embodiment of the presentinvention.

FIGS. 4a-4b illustrate screenshot diagrams of an exemplary homescreen inaccordance with one embodiment of the present invention.

FIGS. 5a-5d illustrate screenshot diagrams of an exemplary homescreenincluding a designated rapid serial visual presentation (“RSVP”) displayarea, which displays textual content associated with a notificationevent using RSVP in accordance with one embodiment of the presentinvention.

FIGS. 6a and 6b illustrate screenshot diagrams of an exemplaryhomescreen including a designated RSVP display area, along with anotification indicating the presence of an attachment or URL inaccordance with one embodiment of the present invention.

FIGS. 7a-7c illustrate screenshot diagrams of an exemplary homescreenincluding a designated RSVP display area in accordance with oneembodiment of the present invention.

FIG. 7d illustrates a screenshot diagram of an exemplary homescreenincluding a designated RSVP display area, along with a notificationindicating the presence of multiple levels of RSVP content in accordancewith one embodiment of the present invention.

FIGS. 7e-7h illustrate screenshot diagrams of an exemplary homescreenincluding a designated RSVP display area displaying the additionallevels of RSVP content indicated in FIG. 7d in accordance with oneembodiment of the present invention.

FIGS. 8a-8c illustrate screenshot diagrams of an exemplary mapsapplication in accordance with one embodiment of the present invention.

FIGS. 9a-9d illustrate screenshot diagrams of an exemplary photoapplication in accordance with one embodiment of the present invention.

FIGS. 10a-10c illustrate screenshot diagrams of an exemplary searchresults page in accordance with one embodiment of the present invention.

FIGS. 11a-11b illustrate screenshot diagrams of an exemplary newswebsite homepage enabled for dense content in accordance with oneembodiment of the present invention.

FIG. 12 is a flow chart diagram of a process of displaying RSVP contenton a homescreen or desktop in accordance with one embodiment of thepresent invention.

FIG. 13 is a flow chart diagram of a process of displaying RSVP contentincluded in file attachments, embedded text, and URLs in accordance withone embodiment of the present invention.

FIG. 14a is a flow chart diagram of a process of displaying RSVP contentwithin an application in accordance with one embodiment of the presentinvention.

FIG. 14b is a screenshot diagram of an exemplary email applicationinterface including a designated RSVP display area in accordance withone embodiment of the present invention.

FIG. 15 is a flow chart diagram of a process of creating an RSVP versionof an online advertisement in accordance with one embodiment of thepresent invention.

FIG. 16 is a flow chart diagram of a process of configuring searchresults with RSVP content in accordance with one embodiment of thepresent invention.

FIG. 17 is a flow chart diagram of a process of embedding RSVP contentin a file in accordance with one embodiment of the present invention.

FIGS. 18a and 18b illustrate screenshot diagrams of an exemplaryspreadsheet application interface illustrating a process of embeddingRSVP content in a spreadsheet document in accordance with one embodimentof the present invention.

FIG. 19 illustrates a screenshot diagram of an exemplary spreadsheetapplication interface including a designated RSVP display area inaccordance with one embodiment of the present invention.

FIGS. 20a-20d show screenshot diagrams of an exemplary video itemlisting page for displaying RSVP content related to video items inaccordance with one embodiment of the present invention.

FIGS. 21a-21g illustrate the playing of a video supplemented by RSVPcontent, the video and associated RSVP content being displayed by anembodiment of the present invention.

FIGS. 22a-22f illustrate a variation on the example illustrated in FIGS.21a-21g , the illustrated video and RSVP content being displayed by anembodiment of the present invention

FIGS. 23a-23i show an RSVP-enabled video preview displayed by anembodiment of the present invention.

FIGS. 24a-24e illustrate an interface of a computerized system inaccordance with an embodiment of the present invention for adding RSVPcontent to a video and creating or editing an RSVP-enabled preview ofthe video.

FIGS. 25a-25k illustrate an augmented reality (AR) applicationsupplemented by RSVP content in accordance with an embodiment of thepresent invention.

FIGS. 26a-26c show examples of various types of alternative knowndisplay devices in which embodiments of the invention might beimplemented.

FIGS. 27a-27d show a time sequence of display of RSVP content in anRSVP-enabled interface implemented in a head-mounted display (“HMD”)device such as, for example, one of the HMD devices shown in FIGS. 26a-26 b.

FIGS. 28a-28b show a process flow implemented by a computer system inaccordance with an embodiment of the present invention for displayingRSVP content within a video player application.

FIGS. 29a-29b show a process flow implemented by a computer system inaccordance with an embodiment of the present invention enabling a userto add RSVP content to a video.

FIG. 30 illustrates a process flow implemented by a computer system inaccordance with an embodiment of the present invention for playing anRSVP-enabled video preview.

FIG. 31 illustrates a process flow implemented by a computer system inaccordance with an embodiment of the present invention for deliveringaugmented reality (“AR”) content elements that are enabled to bedisplayed via RSPV.

FIG. 32 illustrates a process flow implemented by a computer system inaccordance with an embodiment of the present invention for processingand displaying AR content elements that are enabled to be displayed viaRSPV.

DETAILED DESCRIPTION

The present description is made with reference to the accompanyingdrawings, in which various example embodiments are shown. However, manydifferent example embodiments may be used, and thus the descriptionshould not be construed as limited to the example embodiments set forthherein. Rather, these example embodiments are provided so that thisdisclosure will be thorough and complete. Various modifications to theexemplary embodiments will be readily apparent to those skilled in theart, and the generic principles defined herein may be applied to otherembodiments and applications without departing from the spirit and scopeof the disclosure. Thus, the present disclosure is not intended to belimited to the embodiments shown, but is to be accorded the widest scopeconsistent with the principles and features disclosed herein.

Referring now to FIG. 1, there is shown a diagram of an illustrativecomputer system 1000 in accordance with an exemplary embodiment. In thisembodiment, the text display system is implemented on an end user device210, which is configured by computer program product 211 in accordancewith an exemplary embodiment. Computer program product 211 may beprovided in a transitory or non-transitory computer readable medium;however, in a particular embodiment, it is provided in a non-transitorycomputer readable medium (e.g., persistent (i.e., non-volatile) storage,volatile memory (e.g., random access memory), or various otherwell-known non-transitory computer readable mediums).

End user device 210 includes a display 205. In some embodiments, display205 may be configured to accept touch input. Computer program product211 configures device 210 to serially present text in a designated rapidserial visual presentation (“RSVP”) display area 200 on display 205 (forconvenience, referenced herein simply as “DRDA 200”). User device 210may include any type of electronic device capable of controlling textdisplay. Some examples include desktop computers and portable electronicdevices such as mobile phones, smartphones, multi-media players,e-readers, tablet/touchpad, notebook, or laptop PCs, smart televisions,smart watches, head mounted displays, and other communication devices.In some implementations (e.g., a smart phone or e-reader), the display205 may be packaged together with the rest of device 210. However, inother implementations, a separate display device (e.g., a monitor) maybebe attached to device 210. While the illustrated embodiment shows agraphical border around DRDA 200, DRDA 200 simply refers to a region(e.g., a window) on display 205 where text is serially presented inaccordance with an embodiment of the present invention and in particularimplementations, DRDA 200 may or may not be outlined by a graphicalborder.

In one embodiment, user device 210 has typical computer componentsincluding a processor, memory, storage, and an input/output subsystem.In some implementations (e.g., a smart phone or e-reader), user device210 may include a wireless transceiver and one or more input interfacesincluding a touch enabled display, a trackball, keyboard, microphone,smart television controller, etc. In the illustrated embodiment,computer program product 211 is loaded into memory (not separatelyshown) to configure device 210 in accordance with the present invention.In one embodiment, text data may be loaded into memory for textprocessing and display processing by device 210 as will be furtherdescribed herein. Text data loaded into memory for text processing anddisplay processing may be retrieved from persistent storage on a userdevice such as device 210 and/or may be received from one or more servercomputers 101 through a connection to network 102 (e.g., the Internet).One or more server computers 101 may be for example, one or moreadvertiser computers, one or more search engine computers, one or moreweb servers, one or more application servers, etc. In an alternativeembodiment, at least some processing/pre-processing of text data fordisplay in accordance with the principles illustrated herein may becarried out by one or more remote computers such as server computers 101and then sent to end user device 210 for display in DRDA 200 on display205. In such an alternative, some or all of a computer program productsuch as computer program product 211 for implementing an embodiment ofthe present invention may reside on one or more computers such as servercomputers 101 that are remote from end user device 210. In someembodiments, the entire computer program product may be stored andexecuted on remote computers and the results presented within a browserapplication component (e.g. a media player application) of user device210 (browser application and media player application not separatelyshown).

In an embodiment of the invention, text (which includes, for example,strings of characters—e.g., letters, numbers, symbols, etc.—whichconstitute words, numeric figures, and combinations of both withpunctuation marks and symbols—note that words may include wordsrepresented by any human language including, without limitation, wordsrepresented by roman characters, script elements, or pictographiccharacters—) is presented serially (for example, one word at a time)within DRDA 200. As referenced herein, a “display element” will refer toa group of text data that is displayed at one time within DRDA 200. Inother words, display elements are displayed serially. In the primaryembodiment discussed herein, a display element will generally consist ofone word. However, in alternative embodiments, two words may bepresented as a single display element. Also, in the primary embodiment,two words are sometimes part of a single display element such as, forexample, when a number e.g., “9,” is displayed together with a unit,e.g. “feet,” so that, for example, the text “9 feet” may constitute asingle display element and be presented together.

In a conventional RSVP system, each word is centered in the displayarea, and the optimal fixation position shifts as words of differinglengths are sequentially displayed, resulting in saccade movements asthe eyes shift to the optimal fixation position. The reader has torefocus on the display every time a new word appears that is of adifferent length than the previous word. The reader's eyes will movefrom one character to the next to find the optimal position, which isalso referred to as a recovery saccade. In addition, when a longer wordfollows a shorter one, the saccadic movement direction will be fromright to left. When reading text in lines in a traditional paragraphdisplay, most saccadic movement is from left to right so the reader isaccustomed to this type of eye movement. Only occasionally, if theoptimal fixation position is not found directly, the reader may have tomove back from right to left. Thus conventional RSVP forces the readerto experience saccades which are not normal. Conventional RSVPapproaches offer no solution to these problems. In order to prevent orminimize recovery saccades in an RSVP, it is preferable to display eachword such that the optimal fixation position does not shift in thedisplay. The focal point of the reader can then remain fixed on theoptimal fixation position, which is a specific point in each word thatis determined by the total number of characters or width of the word.This optimal recognition position, hereinafter referred to as the “ORP,”can be identified in the display such that the reader's eyes aredirected to focus there as the words are serially presented. An RSVPwhich incorporates an ORP is hereinafter referred to as “ORP-RSVP.” Withan ORP-RSVP, text can then be presented at a faster rate because nosaccades occur during the presentation. In addition, the elimination ofsaccades reduces eye fatigue and makes it more comfortable, resulting ina better reading experience for the user. Embodiments described hereinmay be implemented using conventional RSVP or ORP-RSVP.

In addition, words are rarely greater than 13 characters (according toSigurd, only 0.4% of the words in the English language are longer than13 characters—see Sigurd, B. et al, “Word Length, Sentence Length andFrequency—ZIPF Revisited”, Studia Lingustica 58(1), pp. 37-52, BlackwellPublishing Ltd, Oxford UK, 2004) and therefore, for the vast majority ofwords, it is preferable to limit the number of characters to the rightside of the fixation point to 8 characters. Also, in some embodiments, aword having a length of greater than thirteen characters is divided intofirst and second display elements such that a first portion of the wordis displayed first (along with a hyphen) and then the second portion ofthe word is displayed next. In some embodiments, an empiricallydetermined ORP of each display element is presented at a fixed locationof the DRDA 200. For example, each word of a plurality of words isserially presented and positioned in the display such that the ORP isdisplayed at a fixed display location within DRDA 200 and this enablesrecognition of each word in succession with minimal saccade by thereader. Determining and displaying the ORP for display elements, andpresenting display elements within DRDA 200 is described in more detailin co-pending U.S. application Ser. No. 13/547,982, now U.S. Pat. No.8,903,174, which is hereby incorporated by reference in its entirety. Ingeneral, one method of displaying display elements using the ORP methodinvolves displaying a word such that the fixed display location isoffset so that it is closer to the beginning of the word than to the endof the word, particularly if the word is greater that a certain numberof characters. Research has demonstrated that it is possible to getinformation about a word from up to 4 characters from the left side ofthe fixation position and up to 15 characters to the right side,resulting in a perceptual span of 20 characters, and that the maximumcharacter length of a word without saccade movement is 20 characters.The DRDA 200 can accommodate text of up to 20 characters in lenghtwithout saccades, although it is preferred to limit the display to 13characters for improved comprehension. For example, using one embodimentof an ORP character method, a word four or five characters long would bedisplayed with the second character at the fixed display location. Aword six, seven, eight or nine characters in length would be displayedwith the third character at the fixed display location. A word that isten, eleven, twelve, or thirteen characters long would be displayed withthe fourth character at the fixed display location. Words longer than 13characters may be hyphenated as described above, or, for words 14-20characters in length, for example, displayed with their fifth characterat the fixed display location.

Instead of using whole characters, proportionate positions may be usedin conjunction with a physical width of a word as displayed (e.g., inpixels) to determine, at a pixel level (rather than simply using themiddle of an OPR character), an ORP of a word to place at the fixeddisplay location. In one embodiment, this approach can help account forthe effect of proportionally spaced fonts). In one such a method, alsodisclosed in the above reference co-pending patent application, anoptimal recognition position is calculated as a proportion of a word'slength in pixels and a pixel offset can be used to place such acalculated optimal recognition proportionate positing of the word at thefixed display location.

Therefore, some embodiments of the present invention incorporate amethod for serially displaying text on an electronic display comprisingidentifying an ORP for a plurality of words to be displayed and seriallydisplaying the plurality of words such that the ORP of each word isdisplayed at a fixed display location on the electronic display. In oneembodiment, the ORP is identified as a character in the word. In anotherembodiment, the ORP is identified as a proportionate position relativeto the width of the word in pixels. In some embodiments, visual aids areused to mark the fixed display location (see e.g., hash marks 504 inFIGS. 5a-5d ) and/or an ORP within the word (e.g., different coloredfont).

In one embodiment, configuring text content for RSVP display comprisesparsing text into a plurality of display elements, inserting blankelements at the end of a sentence, and determining a multiplier for eachdisplay element that can be used, along with user selected settingsand/or other display parameters, to determine a display time for eachdisplay element. While, in alternative embodiments, it is possible todisplay each element for the same amount of time, it has beendemonstrated empirically that a longer display time is beneficial forcomprehension of longer words. It has also been demonstrated empiricallythat a longer pause between sentences is beneficial for comprehension oflonger sentences. Further details of certain exemplary systems andmethods for preparing and displaying text using RSVP are described inco-pending U.S. application Ser. No. 13/547,982 referenced above.

FIG. 2a is a block diagram of exemplary software modules included in thecomputer program product loaded on user device 210 of FIG. 1 inaccordance with one embodiment of the present invention. A number ofmodules are shown schematically inside device 210 to indicate modules inthe device associated with text display functionality in accordance withan exemplary embodiment. One or more of the modules may be implementedas part of a programmable microprocessor on the device 210, though theparticular modules are shown separately in this example for clarity ofexplanation. Other arrangements of modules are also contemplated.

User device 210 includes a desktop/homescreen manager 206 to controlvarious elements to be displayed on a homescreen (e.g., a “homescreen”of a smartphone running a mobile operating system such as iOS™,Android™, or Windows Phone™ or a “desktop” display of a PC, laptop,etc., running an operating system such as Windows™, or Mac™ OS; or ahomescreen/desktop screen of an intermediate portable device such asnotepad, touchpad, etc running a corresponding operating system; allreferenced herein as simply a “homescreen” for simplicity). For example,desktop/homescreen manager 206 may control the icons, widgets, tiles,windows, folders, etc. and other information that may be displayed on atraditional computer desktop or on a homescreen (e.g., of a smart phone,tablet, smart television, smart watch, head mounted display, etc.). Aninput manager 212 manages inputs received from one or more inputmechanisms such as a touch-screen, trackball, keyboard, mouse,microphone, eye-tracking, a gesture detector, or other natural interfaceinput detector, smart television controllers, etc. For example, textinput may be provided using a virtual (i.e., touch screen) or physicalkeyboard, mouse, trackball, etc. Alternatively, or in addition, a usermay provide voice/speech input via a microphone, which may then beconverted to text. Various applications 208 (including, for example,applications 208 a, 208 b, and 208 c) may run on the device and mayprovide data to be displayed through desktop/homescreen manager 206.

Various messages (e.g., email, SMS) may be received over a network suchas a wireless communications network connected to the Internet, via awireless interface (not shown). Information received from the network,such as from one or more remote servers, may be provided to theapplications 208 by event manager 202 and information may be passed fromthe applications 208 back to the network. Event manager 202 may managenotification events that are presented to a user, e.g., through display205. For example, event manager 202 may receive notification events fromthe wireless network. Notification events may include, for examplereceipt of text messages, emails, voicemails, social network updates,file transfers, etc. The event manager 202 may in turn forward thenotification events to corresponding applications. For example, an emailnotification may be forwarded to the email application. The applicationmay then instruct the desktop or homescreen manager 206 to displaystatus or notification information to alert the user.

As will be described in further detail below, RSVP library 204 allowsuser device 210 to present display elements using RSVP. In accordancewith an exemplary embodiment, applications 208 a, 208 b, and 208 c,which may be a word processing application, a spreadsheet application, aphoto application, a video application, an augmented realityapplication, a map application, a webpage editor, a browser application,etc., may communicate with RSVP library 204 through a RSVP applicationprogramming interface (API), such as API 214. As will be apparent to oneof skill in the art, an API is an interface used by software componentsto communicate with each other. In one embodiment, each application 208a, 208 b, and 208 c may include application specific RSVP software, suchas RSVP software 209 a, 209 b, and 209 c, respectively, which may allowapplications 208 a, 208 b, and 208 c to detect presence of RSVP content.Upon detecting RSVP content, RSVP software, such as RSVP software 209 a,209 b, and/or 209 c may call RSVP library 204 via the API. In response,RSVP library 204 may display RSVP notification markers/icons, displayDRDA 200, display RSVP content in DRDA 200, etc. In an alternateembodiment, the application specific RSVP software, such as RSVPsoftware 209 a, 209 b, and/or 209 c may instead be included in RSVPlibrary 204. In one embodiment, application specific RSVP software, suchas RSVP software 209 a, 209 b, and 209 c may additionally include logicto allow embedding of textual content within a file. Embedding textualcontent in a file is described in more detail in the description ofFIGS. 17, 18 a and 18 b.

FIG. 2b shows a specific implementation of the embodiment of FIG. 2a inwhich the specific RSVP-enabled applications are video-relatedapplications. Specifically, the implementation shown in FIG. 2b includesvideo playback application 210 a and video editing/annotatingapplication 210 b. In one embodiment, each application 210 a and 210 bmay include application specific RSVP software, such as RSVP software211 a and 211 b, respectively, which may allow applications 210 a and210 b to detect presence of RSVP content. Upon detecting RSVP content,RSVP software, such as RSVP software 211 a and/or 211 b may call RSVPlibrary 204 via the API 214 and RSVP library 204 may respond asdescribed above in the context of FIG. 2a . In an alternate embodiment,the application specific RSVP software, such as RSVP software 211 aand/or 211 b may instead be included in RSVP library 204. In oneembodiment, application specific RSVP software, such as RSVP software211 a and 211 b may additionally include logic to allow embedding oftextual content within a video file. Also, it will be understood bythose skilled in the art, that particular embodiments of suchapplications may be used to enhance a video stream (rather than a file)with added textual content for RSVP display. For example, adding RSVPcontent to may comprise adding that content to a database that relatesthe added content to video stream content so that the video and RSVPcontent may be provided together in a stream for display by anRSVP-enabled media player application. Alternatively, in someimplementations, the RSVP content may be converted in advance to anexisting media player file format (or data stream protocol) and combinedwith the video content (in a file or stream) so that an existing mediaplayer application can display the RSVP content without requiringadditional RSVP specific software. In all embodiments described herein,it will be understood that even if the particular embodiment isdescribed in terms of a “file,” the principles can also be applied todata streams in alternative embodiments.

FIG. 2c shows a specific implementation of the embodiment of FIG. 2aincluding RSVP enabled augmented reality (AR) application 212 withRSPV-specific software 213. Additional components of user device 210that are relevant to AR applications are also shown including cameramanager and image processing application 220, location sensor manager222 (implemented with GPS), direction sensor manager 224, andorientation sensor manager 226. As similarly described in the context ofFIG. 2a , upon detecting RSVP content for AR application 212, RSVPsoftware 213 may call RSVP library 204 via the API 214 and RSVP library204 may respond as described above in the context of FIG. 2a . In analternate embodiment, the application specific RSVP software, such asRSVP software 213 may instead be included in RSVP library 204.

FIG. 2d shows server-side architecture for a server 101 of FIG. 1configured to implement server-side components of an RVSP-enabledaugmented reality (AR) service. Specifically, as shown, server 101includes web server 215, session layer 216, and augmented realityservice application 217. AR application 217 includes RSVP server library218 and API 219. Those skilled in the art will appreciate that thearrangement shown in FIG. 2d could be adapted to provide server-sidecomponents of other RSVP applications such as, for example, a videocontent and/or search application, a photo application, a socialnetworking application, a search engine, news service, a word processingapplication, a spreadsheet application, a photo application, a mapapplication, a webpage editor, or various other types of applicationsthat would benefit from RSVP capabilities. Moreover, in variousembodiments, different divisions of tasks between server and user(client-side) devices can be implemented. In some embodiments, contentis selected and formatted for RSVP presentation at the server and sentto the user device for RSVP display. In other embodiments, content isselected and formatted for RSVP display after it is received at the userdevice. Therefore, in some alternative embodiments, server-side RSVPsoftware will not be separately necessary for particular applications.

FIG. 3 is a schematic diagram of various software modules that may beincluded in RSVP library 204 in accordance with an exemplary embodiment.Though the particular modules are shown separately in this example forclarity of explanation, one or more modules may be combined. Otherarrangements of modules are also contemplated. RSVP library 204 includesRSVP notification markers or icons 302, RSVP parser 304, designated RSVPdisplay area manager 306, RSVP settings 308, RSVP activation listener310, RSVP duration indication 312, and video object recognition unit314. RSVP notification markers or icons 302 are displayed by RSVPlibrary 204 to alert a user of the presence of RSVP content. RSVP parser304 may parse textual data into individual display elements suitable fordisplay in DRDA 200. Designated RSVP display area manager 306 maydisplay DRDA 200 when appropriate user input is received. RSVP settings308 include settings related to displaying display elements using RSVPsuch as, the speed of display, selection of the type of user input toactivate RSVP display, etc. RSVP activation listener 310 detectsappropriate user input which is intended to select a RSVP notificationmarker/icon. RSVP duration indication 312 determines the duration of theRSVP content, which may be displayed, for example adjacent to DRDA. Inone embodiment, the duration may be displayed as an amount of time(e.g., in seconds). Alternatively, or in addition, the duration may bedisplayed graphically as, for example a status bar. As will be apparentto one of skill in the art, that the duration may be displayed variousother ways. In some embodiments, one or more of the modules shown aspart of client side RSVP library 204 may instead or also be provided aspart of a server-side RSVP library such as library 218 of FIG. 2d . Insome embodiments, RSVP library 204 is stored on a server and thenprovided to the user device at the time RSVP content is requested.

FIG. 4a is an illustrative homescreen 400 of a mobile communicationsdevice which has enabled display of textual information using RSVP inaccordance with an exemplary embodiment. Homescreen 400 includes icons402, 404, 406, 408, 410, 412, and 414 corresponding to various differentapplications. Homescreen 400 also includes status area 415 which mayshow notification events or indicators such as battery, signal strength,etc. FIG. 4b is an illustrative homescreen 400 which displays RSVPnotification markers/icons 416, 418 and 420. RSVP notificationmarkers/icons 416, 418 and 420 indicate presence of RSVP content, i.e.,content that is configured to be displayed using RSVP. As shown,notification markers/icons 416, 418 and 420 are overlaid on top of icons404, 402 and 408, respectively. In some embodiments, RSVP notificationmarkers/icons may include a numerical indicator as shown in RSVPnotification markers/icons 416 and 418. The numerical indicators mayindicate the quantity of notifications associated with that application.For example, RSVP notification marker/icon 416 indicates that there aretwo new email messages. As will be apparent to one of skill in the art,the numerical indicators may be incremented as additional notificationevents are received and decremented as notification events are viewed.In some embodiments, notification markers/icons may also be color coded.For example, a notification marker/icon may be a certain color when newnotification events are present, and a different color when allnotification events have been viewed. Alternatively, notificationmarkers/icons may be removed once notification events have been viewed.Although notification markers/icons 416, 418 and 420 are shown overlaidon top of icons 404, 402 and 408, in some embodiments, notificationmarkers/icons may be displayed adjacent to, above, or below icons 404,402 and 408. In other embodiments, notification markers/icons 416, 418and 420 may be displayed in other areas of homescreen 400. For example,notification markers/icons may be displayed in status area 415. In otherimplementations, such as on desktop or laptop PCs, notificationmarkers/icons may be displayed in the taskbar area typically located inthe bottom right corner. In some embodiments, application specificnotification markers/icons, DRDA, etc. may be implemented. For example,a social networking application may employ customized RSVP notificationmarkers/icons and/or DRDA such that the RSVP notification markers/iconsand/or DRDA are consistent with the application UI.

In yet another embodiment, for certain applications, notificationmarkers/icons may not be displayed. For example, for SMS and/or emailapplications, where notification events typically include textualcontent, RSVP display may be enabled by default such that contentassociated with all notification events for these applications may bedisplayed using RSVP upon user selection of a notification event or uponuser interaction with a selectable item (e.g., an email or a textmessage). As another example, even when an underlying application'scontent is non-textual, the content items may be known to be associatedwith textual descriptions. For example, a video application icon on ahomescreen may be enabled to display textual descriptions, using RSVP,of new or featured video items by default from a home screen such thatthe video application's icon is selectable for RSVP display of suchdescriptions by default without the need for a separate RSVP-specificnotification marker.

FIGS. 5a-5d illustrate screenshot diagrams of an exemplary homescreenincluding a designated RSVP display area, which displays textual contentassociated with a notification event using RSVP in accordance with oneembodiment of the present invention. As shown in FIGS. 5a-5d , upon userselection of a notification marker/icon, textual content associated withthe corresponding notification event is displayed using RSVP in DRDA502. The user may select a notification marker/icon a number ofdifferent ways. For example, in a smartphone with a touch enableddisplay implementation, the user may touch and hold the notificationmarker/icon. Alternatively, the user may tap the notificationmarker/icon, swipe across the notification marker/icon, etc. In oneembodiment, user selection of notification markers/icons may bedetermined by eye-tracking, gesture detection, or other naturalinterface input detection. As will be apparent to one of ordinary skillin the art, various different user input methods may be used to selectthe notification marker/icon. In other implementations, such as on adesktop or laptop PC, smart television, or on devices without a touchenabled display, user selection of notification markers/icons may beprovided using, for example a keyboard, mouse, trackball, eye-tracking,smart television remote controller, etc. It should be noted that userselection of notification markers/icons does not necessarily requireclicking or tapping a particular notification marker/icons. Userselection may be made simply by “rolling” or “hovering” over anotification marker/icon. In one embodiment, the desired method may beselected in RSVP settings 308 (see FIG. 3).

In the example illustrated in FIGS. 5a-5d , if the user selectsnotification marker/icon 418, DRDA 502 may be displayed on homescreen400, and DRDA 502 may display, using RSVP, the first of two SMSmessages, which reads “Martina: please call me.” It should be noted thatFIGS. 5a-5d illustrate time sequence snapshots of the content displayedin DRDA 502. For example, FIG. 5a shows that DRDA 502 displays“Martina:” at time 1, FIG. 5b shows that DRDA 502 displays “please” atsecond time, FIG. 5c shows that DRDA 502 displays “call” at third time,and FIG. 5d shows that DRDA 502 displays “me” at fourth time. DRDA 502may display an ORP of each word at a fixed display location between hashmarks 504. As previously discussed, an empirically determined ORP ofeach display element may be presented at a fixed location of DRDA 502,the fixed display location being between hash marks 504. It should benoted that hash marks 504 are shown in some but not all subsequentfigures to avoid overcomplicating the figures, and particularembodiments of the invention may be implemented using either ORP-RSVP orconventional RSVP. In addition, although DRDA 502 is shown displayed ontop of the display, DRDA 502 may be positioned in any portion of thedisplay. Although the embodiment illustrated in FIGS. 5a-5d showsdisplay of textual content associated with a notification event usingRSVP on a homescreen, in some embodiments, content may be displayedusing RSVP on a screen other than the homescreen, such as, for example,a “lock screen.” As will be apparent to one of skill in the art,notification events are often displayed on a “lock screen.” In such aninstance, selecting the notification(s) on the lock screen may result incontent associated with the notification event being displayed in a DRDAusing RSVP on the “lock screen.”

FIGS. 6a and 6b illustrate screenshot diagrams of an exemplaryhomescreen including a designated RSVP display area, along with anotification indicating the presence of an attachment or URL inaccordance with one embodiment of the present invention. As shown, theRSVP content may include one or more file attachments and/or URLs. Ifthere is an attachment and/or URL included in the RSVP content, anotification indicating the presence of an attachment and/or URL may bedisplayed with the DRDA. In one embodiment, a thumbnail imagecorresponding to the attachment or URL may be displayed in addition to,or instead of, the notification. FIG. 6a illustrates DRDA 602 with anotification 604 indicating the presence of a map attachment. Also shownis thumbnail 603, which corresponds to the map attachment. Althoughthumbnail 603 is shown displayed adjacent to DRDA 602, thumbnail 603 maybe displayed anywhere on the display. Various types of attachments maybe included with RSVP content. For example, in addition to maps, photos,documents (e.g., PDFs), etc. may also be attached to RSVP content. FIG.6b illustrates an example with a notification 606 indicating that a URLis included with the RSVP content. As discussed above in the example ofa map attachment, a thumbnail image (not shown in FIG. 6b )corresponding to the URL may be displayed in addition to, or instead of,notification 606. It should be noted that although notifications 604 and606 are displayed underneath DRDA 602, notifications 604 and 606 may bedisplayed adjacent to, above, or below DRDA 602. Alternatively, thenotifications may be displayed in any other location on the display. Insome embodiments, the content of a webpage corresponding to the URL maybe displayed using RSVP in DRDA 602. In other embodiments, anapplication corresponding to the attachment may be launched to open theattachment and/or URL. For example, a photo application may be launchedto open a photo, or a web browser may be launched to open a URL includedwith RSVP content. The process of displaying content included in anattachment and/or content of a webpage corresponding to a URL includedwith RSVP content is described in more detail in the description of FIG.12.

FIGS. 7a-7c illustrate screenshot diagrams of an exemplary homescreenincluding a designated RSVP display area, and FIG. 7d illustrates ascreenshot diagram of an exemplary homescreen including a designatedRSVP display area, along with a notification indicating the presence ofmultiple levels of RSVP content in accordance with one embodiment of thepresent invention. It should be noted that FIGS. 7a-7d illustrate timesequence snapshots of the content displayed in DRDA 702. For example,FIG. 7a shows that DRDA 702 displays “Meet” at time 1, FIG. 7b showsthat DRDA 702 displays “John” at second time, FIG. 7c shows that DRDA702 displays “and” at third time, and FIG. 7d shows that DRDA 702displays “Kevin” at fourth time. In one embodiment, RSVP content mayinclude multiple or “cascading” levels of RSVP content. For example,RSVP content may itself include further RSVP content. The presence ofadditional levels of RSVP content (sometimes referenced herein as“sub-content”) may be indicated by, for example displaying anotification similar to the notifications used to indicate the presenceof an attachment and/or URL (see FIGS. 6a and 6b ). FIGS. 7a-7dillustrate DRDA 702 which displays the message “Meet John and Kevin.”However, this message includes additional RSVP content as indicated bynotification 704 in FIG. 7d , which displays “John” and “Kevin.” “John”and “Kevin” are selectable items within notification 704 which representfurther embedded RSVP content. For example, if the user selects “John”,the RSVP content associated with “John” may be displayed.

FIGS. 7e-7h illustrate screenshot diagrams of an exemplary homescreenincluding a designated RSVP display area displaying the additional levelof RSVP content associated with “John” in accordance with one embodimentof the present invention. DRDA 702 displays “John's business is Acme.”As discussed above, FIGS. 7e-7h illustrate time sequence snapshots ofthe content displayed in DRDA 702. The message “John's business is Acme”may further include embedded RSVP content, as indicated by “Acme” innotification 704. If the user selects “Acme,” the RSVP contentassociated with “Acme” may be displayed (not separately shown).Similarly, if the user selects “Kevin,” RSVP content associated with“Kevin” may be displayed in DRDA 702 (not separately shown). It shouldbe noted that although additional RSVP content was embedded in “John”and “Kevin,” RSVP content may also be embedded in phrases, sentences,paragraphs, etc. For example, “John's business is Acme” may have beenembedded in “Meet John” instead of just “John.”

FIGS. 8a-8c illustrate a screenshot diagram of an exemplary mapsapplication in accordance with one embodiment of the present invention.RSVP content may be implemented in virtually any application. Forexample, FIG. 8a shows a maps application in which the presence of RSVPcontent is indicated by notification markers/icons 804, 806 and 808.Notification 802 may be included to indicate the quantity ofnotification markers/icons displayed on the map, and to notify the userof “global” RSVP content corresponding to the whole map (as opposed toRSVP content corresponding to specific locations on the map, which arespecified by markers/icons 804, 806 and 808). For example, notification802 may have associated with it RSVP content such as “Check out thesethree restaurants.” Thus, if notification 802 is selected by the user,DRDA 810 may display “Check out these three restaurants” (this messageis not separately shown in the figures and DRDA 810 is not separatelyshown in FIG. 8a ). It should be noted that although notification 802 isshown in the top-right corner, notification 802 may be displayedanywhere on the display. As shown, notification markers/icons 804, 806and 808 include numerical indicators. In one embodiment, the numericalindicators may be used to denote quantity. In other embodiments, thenumerical (or, alternatively, alphabetical) indicators may be used toindicate a preferred viewing order. For example, notificationmarkers/icons 804, 806 and 808 may be used to indicate presence of RSVPcontent related to restaurants at specified locations on the map, andthe numerical indicators alert the user to view the notifications in thespecified order. If the user selects notification marker/icon 804, DRDA810 may be displayed as shown in FIG. 8b . As shown in FIGS. 8b and 8c ,DRDA 810 may display “Great burgers!,” which is associated withnotification marker/icon 804, using RSVP. It should be noted that FIGS.8b-8c illustrate time sequence snapshots of the content displayed inDRDA 810. For example, FIG. 8b shows that DRDA 810 displays “Great” attime 1, and FIG. 8c shows that DRDA 810 displays “burgers” at secondtime. In one embodiment, DRDA 810 may automatically display RSVP contentassociated with markers 804, 806 and 808 in sequential order after userselection of notification 802. For example, if the user selectsnotification 802, DRDA 810 may display “Check out these threerestaurants” and then automatically display “Great burgers!”,(associated with marker 804) and then automatically display RSVP contentassociated with markers 806 and 808. In one alternative, the map isautomatically moved to center the marker for which text is currentlybeing displayed by RSVP.

FIGS. 9a-9d illustrate screenshot diagrams of an exemplary photoapplication in accordance with one embodiment of the present invention.As shown in the example of FIGS. 9a-9d , RSVP content may be embedded ina photo. As shown in FIG. 9a , notification 902 may be displayed toindicate the quantity of notification markers/icons, and to notify theuser of “global” RSVP content associated with the whole photo (asopposed to RSVP content associated with specific locations on the photo,which are specified by markers/icons 904, and 906). For example,notification 902 may have associated with it RSVP content such as “Theseare my friends.” Thus, if notification 902 is selected by the user, DRDA908 may display “These are my friends” using RSVP (this message is notseparately shown in the figures and DRDA 908 is not separately shown inFIG. 9a ). It should be noted that although notification 902 is shown inthe top-right corner, notification 902 may be displayed anywhere on thedisplay. Notification markers/icons 904 and 906 may be displayed toindicate presence of “local” RSVP content associated with the locationon the photo specified by the notification marker/icon. For example, asshown in FIG. 9b , upon user selection of notification marker/icon 906,DRDA 908 may be displayed, and “Martina,” which is associated with thelocation on the photo specified by notification marker/icon 906 may bedisplayed in DRDA 908. Similarly, as shown in FIGS. 9c and 9d , uponuser selection of notification marker/icon 904, DRDA 908 may bedisplayed, and “Martina's boyfriend,” which is associated with thelocation on the photo specified by notification marker/icon 904 may bedisplayed using RSVP in DRDA 908. It should be noted that FIGS. 9c-9dillustrate time sequence snapshots of the content displayed in DRDA 908.For example, FIG. 9c shows that DRDA 908 displays “Martina's” at time 1,and FIG. 9d shows that DRDA 908 displays “boyfriend” at second time. Theprocess of embedding RSVP content in photos is described in more detailin the description of FIG. 17. In one embodiment, DRDA 908 mayautomatically display RSVP content associated with markers 904, and 906in sequential order after user selection of notification 902. Forexample, if the user selects notification 902, DRDA 908 may display“These are my friends” and then automatically display “Martina”(associated with marker 906), and then automatically display “Martina'sboyfriend” (associated with marker 904). In a particular embodiment,when RSVP content is played automatically for multiple markers, eachmarker is visually highlighted (e.g., bolded) while the RSVP contentassociated with that marker is being displayed.

FIGS. 10a-10c illustrate screenshot diagrams of an exemplary searchresults page in accordance with one embodiment of the present invention.Another example in which RSVP content may be employed is in the displayof search results, including sponsored search results, displayadvertisements, etc. FIG. 10a shows a screenshot of a search resultspage for a search query “hotel in paris.” As shown, one or more searchresults corresponding to the search query may be displayed. One or moreof the search results (or sponsored search results, displayadvertisements, etc.) may be displayed with a notification marker/iconindicating the presence of RSVP content associated with that searchresult (or sponsored search results, display advertisements, etc.). Forexample, notification markers/icons 1002 and 1004 may be displayedadjacent to search results 1006 and 1008, respectively, to indicatepresence of RSVP content associated with those search results. Searchresult 1006 may be a sponsored search result. In some embodiments,notification markers/icons 1002 and 1004 may be displayed above, below,or in any other location on the display. In one embodiment, advertisersand/or search engine providers may choose to embed specifically chosencontent as RSVP content in the search results in order to include moreinformation than what is displayable in the provided space. In anotherembodiment, content from a webpage may be embedded as RSVP content. Forexample, in the case of sponsored search result 1006, the advertiser mayprovide content which the advertiser wishes to be embedded as RSVPcontent (e.g., “Free Breakfast” as shown in FIGS. 10b and 10c ).However, in some embodiments, like in the case of search result 1008,content from the corresponding webpage (e.g., HTML text) may be embeddedas RSVP content. The process of embedding RSVP content in search results(or sponsored search results, display advertisements, etc.) is describedin more detail in the description of FIG. 16.

Upon user selection of notification marker/icon 1002, DRDA 1010 may bedisplayed as shown in FIGS. 10b and 10c , and “Free breakfast” may bedisplayed using RSVP in DRDA 1010. It should be noted that FIGS. 10b-10cillustrate time sequence snapshots of the content displayed in DRDA1010. For example, FIG. 10b shows that DRDA 1010 displays “Free” at time1, and FIG. 10c shows that DRDA 1010 displays “breakfast” at secondtime.

FIGS. 11a-11b illustrate screenshot diagrams of an exemplary newswebsite homepage configured with RSVP content in accordance with oneembodiment of the present invention. In one embodiment, RSVP content maybe employed on websites to advantageously increase content density. Thisallows websites that provide large quantities of content such as, forexample, news portals and ecommerce sites, to benefit from fasterselection and reading of content by the visitor, especially oncommunication devices which have small displays. For instance, in theexample of a news website, news headlines 1102, 1104, 1106, 1108, 1110,1112, and 1114 may be presented as shown in FIG. 11a . As shown in FIG.11b , upon user selection of one of headlines 1102, 1104, 1106, 1108,1110, 1112, and 1114, DRDA 1116 may be displayed, and the contents ofthe corresponding story may be displayed using RSVP in DRDA 1116. In oneembodiment, the content of the news story may be configured using a<meta> HTML tag. For example, the story may be configured using <metaname=“RSVP” content=“RSVP content goes here”>. Thus, when the browserapplication opens the page, it may search for the <meta name=“RSVP”>tag, and display the text included in the content parameter using RSVP.

FIG. 12 is a flowchart diagram of process 1200 for displaying RSVPcontent on a homescreen in accordance with one embodiment of theinvention. A new notification event is received at step 1202. Aspreviously described, a new notification event may include receipt of anew email message, SMS message, file transfer, etc. In one embodiment,the notification event may have been received as a result of a “push”notification. Alternatively, the notification event may have beenreceived in response to a request from user device 210. It may bedetermined in step 1204 if the notification event is displayable usingRSVP. In one embodiment, this determination includes determining if thecorresponding application allows display of content using RSVP. Forinstance, as a result of user settings, certain applications may havedisabled or may not allow displaying content using RSVP. If it isdetermined that the notification event is not displayable using RSVP,the notification event is forwarded to the corresponding application instep 1206, which may handle the notification event in the conventionalmanner. For example, if the notification is for a SMS message, thenotification is forwarded to the SMS application, if it for an emailmessage, it is forwarded to the email application, etc.

However, if the notification event is determined to be displayable usingRSVP, a notification marker/icon may be displayed in step 1208. In oneembodiment, the notification marker/icon may be overlaid on top of theicon representing the corresponding application (see, FIG. 4b ). Inother embodiments the notification marker/icon may be displayed adjacentto, above or below the icon representing the corresponding application.As previously described, the notification marker/icon may also include anumerical indicator, which may correspond to the number of notificationevents for the corresponding application. The notification marker/iconmay be selected in response to user input received at step 1210. Theuser input may include, for example a touch and hold of the notificationmarker/icon, a tap of the notification marker/icon, a selection of thenotification marker/icon using a mouse or trackball, etc. As previouslydiscussed, in some embodiments, a notification marker/icon may beselected by simply “hovering” over the notification marker/icon. Inresponse to user selection of the notification marker/icon, thedesignated RSVP display area (DRDA) may be displayed in step 1212.Textual content associated with the notification event may be displayedusing RSVP in the DRDA in step 1214. In one embodiment, the textualcontent associated with the notification event may be displayed in theDRDA as long as appropriate user input is present. For example, in thecase where appropriate user input is a press and hold of thenotification icon/marker, if the user releases the notificationicon/marker, the DRDA stops displaying the RSVP content and is thenhidden. In one embodiment, the amount and type of content that getsdisplayed in DRDA may be defined or selected in RSVP Settings 308 (FIG.3). For example, in the case of an email, it may be selected that thesender name, subject of the email and the body of the email may bedisplayed using RSVP in DRDA. Alternatively, it may be selected thatonly the sender name and subject of the email be displayed using RSVP inDRDA. In some embodiments, RSVP content may contain additional embeddedRSVP content as illustrated in, for example FIGS. 7a-7h . It may bedetermined in step 1216 if additional embedded text is present. If so, anotification (e.g., notification 704 in FIG. 7d ) may be displayed instep 1218. In response to user input received at step 1220, theadditional embedded text may be displayed in DRDA using RSVP. In someembodiments, steps 1218 and 1220 may be optional. In other words, allavailable RSVP content may be automatically displayed sequentiallywithout awaiting user input.

FIG. 13 is a flowchart diagram of process 1300 for displaying embeddedtext (see e.g., FIG. 7d ), an attachment and/or URL included with RSVPcontent in accordance with one embodiment of the invention. Uponcompletion of displaying RSVP content, which may or may not have beenassociated with a notification event, it is determined in step 1302whether the RSVP content includes sub-content such as additionalembedded text, an attachment and/or a URL. If the RSVP content isdetermined to not include embedded text, an attachment and/or a URL, theDRDA is hidden in step 1304. If the RSVP content is determined toinclude embedded text, an attachment and/or a URL, it is determined instep 1303 whether embedded text, a file attachment or a URL is included.If it is determined that a file attachment is included, an indication ofthe presence of an attachment is displayed in step 1305 (as shown inFIG. 6a ). The file attachment may be, for example a photo, a map, adocument, etc. User input indicating selection of the file attachment isreceived in step 1307. In step 1309, the DRDA is hidden and theapplication associated with the file attachment is launched to open theattachment. For example, if the file attachment is a photo, a photoviewer application may be launched to open the photo. In one embodiment,the file attachment itself may contain embedded RSVP content. Theprocess of opening a file which may contain RSVP content and displayingthe RSVP content within an application interface is described in moredetail in the description of FIG. 14 a.

If it is determined in step 1303 that a URL is included in the RSVPcontent, an indication of the presence of a URL is displayed in step1306 (as shown in FIG. 6b ). User input indicating selection of the URLis received in step 1308. In one embodiment, the webpage correspondingto the URL may contain RSVP content. As previously described, RSVPcontent may be embedded in a webpage using, for example a <meta> HTMLtag. It is determined in step 1310 if the webpage corresponding to theURL contains RSVP content. If it is determined that the webpage does notcontain RSVP content, the DRDA is hidden and a browser application islaunched in step 1312 to display the webpage corresponding to the URL.If, however, the webpage is determined to contain RSVP content, the RSVPcontent is displayed in the DRDA using RSVP in step 1314.

If it is determined in step 1303 that embedded text is included in theRSVP content, an indication of the presence of embedded text isdisplayed in step 1316 (as shown in FIG. 7d ). User input indicatingselection of the embedded text is received in step 1318. In step 1320,the RSVP content is displayed in the DRDA using RSVP.

FIG. 14a is a flowchart diagram of process 1400 for displaying, withinan application, RSVP content that is embedded in a file in accordancewith one embodiment of the present invention. For example, process 1400may be used by a photo application to display a photo which containsembedded RSVP content, or by a browser application to display anadvertisement which contains embedded RSVP content, etc. In step 1402,the file may be opened by the application. In step 1404, it may bedetermined if the file contains RSVP content (e.g., content that isconfigured to be displayed using RSVP). In one embodiment, thisdetermination may be made by RSVP library 204 (FIG. 2). The applicationmay request RSVP library 204 to determine if the file includes RSVPcontent. In one embodiment, RSVP library 204 may detect the presence ofRSVP content by searching the contents of the notification event formeta data, such as a <meta> tag in HTML pages, EXIF data in photos, etc.In another embodiment, the application may include application specificRSVP software (e.g., 209 a-209 c in FIG. 2), which may determine if thefile contains RSVP content. Although FIG. 14a illustrates opening a filewithin an application interface, the same logic is applicable toinitiating a stream of data within a browser or other application.

If it is determined that the file contains no RSVP content, theapplication may proceed as normal in step 1406. For example, if the fileis a photo, and it does not contain RSVP content, the application maysimply display the photo normally. If however, it is determined that thefile contains RSVP content, the application requests display ofnotification markers/icons in appropriate locations in step 1408. In theexample of a photo, the notification markers/icons may need to bedisplayed in specific locations on the photo. In one embodiment, theapplication may request, for example, RSVP library 204 to display thenotification markers/icons. In another embodiment, the application mayrequest the operating system to display the notification markers/icons.In yet another embodiment, the application may obtain the notificationmarkers/icons from RSVP library 204, and display the notificationmarkers/icons. In an alternate embodiment, step 1408 of displayingnotification markers/icons is optional. A user device may be configured(e.g., using RSVP settings 308 in FIG. 3) such that certain applicationsmay display all content using RSVP. For example, applications such as anemail application, a SMS application, etc. which primarily receivetextual data, may be configured to display all content using RSVP. Insuch instances, displaying a notification marker/icon indicating thepresence of RSVP content is not necessary as further described withrespect to FIG. 14 b.

Continuing with the description of FIG. 14a , upon receiving user inputcomprising selection of a particular notification marker/icon at step1410, the application may request display of the DRDA in step 1412. Inone embodiment, the application may request, for example RSVP library204 to display the DRDA. In another embodiment, the application mayrequest the operating system to display the DRDA. In yet anotherembodiment, the application may itself (e.g., using RSVP software 209a-209 c in FIG. 2) display the DRDA. The RSVP content associated withthe user selected notification marker/icon may be displayed in the DRDAusing RSVP in step 1414. It is determined in step 1416 if additionalRSVP content is present. If so, the additional RSVP content may beautomatically displayed sequentially. For example, as shown in FIGS.7a-7c and 8a-8d , if there are multiple notification markers/icons, DRDAmay automatically sequentially display the RSVP content associated witheach marker/icon. In another embodiment, user input may be requestedbefore displaying RSVP content associated with each marker/icon. Aspreviously discussed, the RSVP content may include one or moreattachments. After displaying RSVP content, process 1300 may be used todisplay any attachments included with the RSVP content.

FIG. 14b is a screenshot diagram 1450 of an exemplary email applicationon a mobile device in accordance with one embodiment of the presentinvention. Screenshot diagram 1450 shows a list of email items 1454which are displayed in a user's inbox. In one embodiment, process 1400,as discussed above, may be employed by the email application to displaythe content of selected emails using RSVP. A user may select an email byfor example, “touching and holding” an email item 1454. Upon userselection of an email, DRDA 1452 may be displayed, and the content ofthe email may be displayed in DRDA 1452 using RSVP. In one embodiment,the content displayed in DRDA 1452 may include the body of the email.Although notification markers/icons 1456 are shown, displayingnotification markers/icons 1456 is optional. As discussed above, a userdevice may be configured (e.g., using RSVP settings 308 in FIG. 3) suchthat certain applications may display all content using RSVP. An emailapplication, which primarily receives textual data, may display allcontent using RSVP. In such instances, displaying a notificationmarker/icon indicating the presence of RSVP content is not necessary. Inone such example, an application may simply list item titles (e.g.,emails) without using RSVP notification markers/icons and then, uponreceipt of a particular user interface action (e.g., a tap and hold)with respect to an item, the application will display content of thatitem using RSVP in a DRDA while continuing to display at least a portionof the list of other items. As another example, even when an underlyingapplication's content is non-textual, the content items may be known tobe associated with textual descriptions. For example, a videoapplication may be enabled to display a list of video items that areassociated with RSVP-displayable textual descriptions and the videoitem's representation is selectable using a particular user interfaceaction for RSVP display of such descriptions by default without the needfor a separate RSVP-specific notification marker.

FIG. 15 is a flow chart diagram of process 1500 for creating an RSVPversion of an online advertisement in accordance with one embodiment ofthe present invention. Online advertisers, who are looking to increasethe amount of content that is presented to users in a given amount ofscreen space, may benefit from incorporating RSVP content into theiradvertisements. Textual information to be incorporated into anadvertisement may be received from an advertiser in step 1502. It may bedetermined in step 1504, whether the advertiser would like to embed thetextual information as RSVP content in the advertisement. If it isdetermined that the advertiser does not want to include RSVP content inthe advertisement, a non-RSVP version of the advertisement may becreated in step 1506. If it is determined that the advertisement is toinclude RSVP content, an RSVP version of the advertisement may becreated in step 1508. The RSVP content may be configured using, forexample meta-data tags as previously discussed. In step 1510, a RSVP ornon-RSVP version of the advertisement may be saved on one or moreservers. One or more servers may receive a request for an advertisementin step 1512. In one embodiment, the request may be from a browserapplication (e.g., for a banner advertisement). In another embodiment,the request may be from a desktop or mobile application which displaysadvertisements within its interface. In response to the request, theadvertisement may be transmitted to the application in step 1514. In oneembodiment, when the advertisement is received by the application,process 1400 may be used to display the advertisement. In oneembodiment, the advertisement may be displayed with one or morenotification markers/icons similar to those shown in FIG. 4b, 8a, 9a ,or 10 a to indicate the presence of content that is displayable usingRSVP. The notification markers/icons may be overlaid on top of theadvertisement, or may be displayed adjacent to, above, or below theadvertisement.

In addition to display advertisements, RSVP content may also beincorporated in, for example sponsored search results. FIG. 16 is a flowchart diagram of a process 1600 for configuring search results with RSVPcontent in accordance with one embodiment of the present invention. Oneor more servers may receive a search query in step 1602. In response tothe search query, at least one search result corresponding to the searchquery may be configured with content that is displayable using RSVP instep 1604. In step 1606, the at least one search result may betransmitted to a browser application for display in the browserapplication. In one embodiment, the search result(s) may be displayed asshown in FIGS. 10a -10 c.

In some embodiments, users may embed RSVP content into files such as forexample, photos, presentations, maps, etc. FIG. 17 is a flow chartdiagram of a process 1700 for embedding RSVP content in a file (e.g.,map, photo, diagram, presentation, word processing document,spreadsheet, etc.) in accordance with one embodiment of the presentinvention. In step 1702, displayable contents of a file may be displayedby the appropriate application. For example, a diagram may be displayedin a drawing application (e.g., Microsoft Visio™), or a presentation maybe displayed in a presentation application (e.g., MicrosoftPowerPoint™). In some embodiments, the application may be a desktopapplication while in other embodiments, the application may be a mobileapplication, or a web based application accessible via a browser. A usermay specify whether to add “global” RSVP content or “local” RSVP contentin step 1704. In one embodiment, an interface which allows the user tochoose between “global” and/or “local” RSVP content may be displayed. Insome embodiments, both “local” and “global” RSVP content may beembedded. As used herein, “global” content refers to RSVP contentassociated with the file (e.g., photo, map, presentation, and/ordiagram) in its entirety. In the case of a map, in the example shown inFIG. 8b , notification 802 indicates presence of “global” RSVP content,such as “Check out these three restaurants” and is associated with thewhole displayed map. By contrast, notifications 804, 806, and 808indicate presence of “local” RSVP content associated with each locationspecified by notifications 804, 806, and 808. Similarly, in the case ofa photo, in the example shown in FIG. 9a , notification 902 indicatespresence of “global” RSVP content, such as “These are my friends,” andis associated with the whole photo. By contrast, notifications 904 and906 indicate presence of “local” RSVP content associated with eachlocation specified by notifications 904 and 906. In another embodiment,“global” content may include a description of the file's contents. Insuch an embodiment, selecting a file (e.g., by clicking on, or“hovering” over the file's icon or name) that includes embedded “global”RSVP content from a file management interface (e.g., Windows Explorer™)or a homescreen may result in the “global” content being displayed usingRSVP. It should be noted that in such an embodiment, viewing the“global” RSVP content does not require opening the file with anassociated application.

If the user chooses to add “global” RSVP content, an interface whichallows the user to input textual content, which will be embedded as“global” RSVP content is displayed in step 1706. In the embodimentdescribed above where “global” content includes a description of thefile, the “global” content may be provided in the metadata of the file.For example, a user may right-click the file icon, select an option toenter an RSVP description for the file, and input the textual content.

User input including textual content may be received in step 1708. Inembodiments where text input is required, text input may be providedusing a virtual (i.e., touch screen) or physical keyboard, mouse,trackball, etc. Alternatively, or in addition, the user may providevoice/speech input via a microphone. The voice/speech input may then beconverted to text. If the user chooses to add “local” RSVP content, aninterface which allows the user to specify location(s), element(s),and/or text selection, and add the corresponding textual content, whichwill be embedded as “local” RSVP content is displayed in step 1710. Forexample, the user may specify locations on photos, maps, etc., specifyelements or objects in photos, diagrams and presentations, etc. Userinput including specified location(s), element(s), and/or textselection, and add the corresponding textual content may be received instep 1712. In one embodiment, the interface may also allow selection oftext such that RSVP content may be associated with the selected text. Inthe case of a photo, in addition to specifying locations on the photo,the user may select areas of the photo, such as, for example a face of aperson. The location(s), element(s), and/or text selection informationand the corresponding textual content may be saved in step 1714. In oneembodiment, location(s), element(s), and/or text selection informationand the corresponding textual content may be saved in the meta data ofthe file. For example, in the case of a photo, the location(s),element(s), and/or text selection information and the correspondingtextual content may be saved in the EXIF data. In one embodiment, thetextual content may be saved as plain text, in which case, the textualdata may be converted to RSVP content (e.g., text that is configured tobe displayed using RSVP) prior to displaying. In other embodiments, thetextual content may be converted to RSVP content, and the RSVP contentmay be saved in the meta data of the file.

FIGS. 18a and 18b illustrate screenshot diagrams of an exemplaryspreadsheet application interface illustrating a process of embeddingRSVP content in a file such as for example, a spreadsheet document inaccordance with one embodiment of the present invention. RSVP contentmay be embedded as a comment on any cell in a spreadsheet document. Toembed RSVP content, a cell 1801 may be selected as shown in FIG. 18a .Once selected, a user may click on for example, “New Comment,” icon 1802or may right-click on the cell. Right-clicking on the cell may display amenu, from which “Insert Comment” may be selected. Selecting “NewComment” or “Insert Comment” may result in the display of input field1804 as shown in FIG. 18b . The user may specify a sequence number forthe comment, if any, and provide the text for the comment in input field1804. The text entered into input field 1804 is displayable using RSVP.In one embodiment, the content of every cell and/or every commentcorresponding to a cell may be displayable using RSVP. Optionally, anotification marker/icon may be displayed indicating the presence ofcontent or a comment including RSVP content. A process similar to thatshown in FIGS. 18a and 18b may be used to embed RSVP content in a wordprocessing document. For example, text may be selected, and acorresponding comment may be input by a user. The comment may then beembedded in the word processing document, and may subsequently bedisplayed in a DRDA using RSVP upon user selection of the comment.

FIG. 19 illustrates a screenshot diagram of an exemplary spreadsheetapplication interface including a designated RSVP display area inaccordance with one embodiment of the present invention. Screenshot 1900illustrates a spreadsheet application interface including DRDA 1904displaying RSVP content that was embedded in the spreadsheet document.As shown, notification marker/icon 1902 indicates presence of RSVPcontent associated with the corresponding cell. In addition,notification marker/icon 1902 may also include a numerical indicator(not shown). Upon user selection of notification icon/marker 1902, DRDA1904 may be displayed, and the corresponding RSVP content may bedisplayed in DRDA 1904. In one embodiment, the spreadsheet applicationmay be configured to allow the content and/or comments corresponding toall cells to be displayed using RSVP whether or not an author hasspecially embedded content designated for RSVP display. In such aninstance, notification markers/icons indicating presence of RSVP contentmay not be displayed. A user may for example, “touch and hold” a cell,and as a result, DRDA 1904 may be displayed. The corresponding contentand/or comment may then be displayed using RSVP in DRDA 1904.

FIGS. 20a-20d show screenshot diagrams of an exemplary video itemlisting window in accordance with one embodiment of the presentinvention. FIG. 20a shows a web application window 2000 (which maycorrespond to a web page) including a list of featured video itemswithin a browser-based video content application. Video item listingswith links to corresponding video content are provided by many differentknown video content applications, e.g., YouTube™. However, in theillustrated embodiment, the video item listing is supplemented byproviding access to related RSVP content. As shown, a list is displayedof featured video item representations 2001, 2002, 2003, and 2004.Markers/icons 2011, 2012, 2013, and 2014 are displayed with,respectively, video item representations 2001, 2002, 2003, and 2004.Each marker indicates the presence of RSVP content associated with thecorresponding video item.

As shown in FIG. 20b , upon user selection of notification marker 2011,DRDA 2020 may be displayed and, as shown in FIGS. 20b-20d , RSVP contentrelated to video item 2001 is displayed in DRDA 2020. FIGS. 20b-10d showtime-sequence snapshots of the content displayed in DRDA 2020. Forexample, FIG. 20b shows that DRDA 2020 displays the word “People” at afirst time; FIG. 20c shows that DRDA 2020 displays the word “have” at anext time; and FIG. 20d shows that DRDA 2020 displays the word “played”at third time.

As shown in FIGS. 20a-20d , this embodiment allows a user to receivemore information about a particular video within the context of asmall-screen display before deciding whether to leave window 2000 andnavigate to a new window related to a particular video item. Theapplication interface is able to continue displaying window 2000including the illustrated list of video items while displayingadditional information about a selected video item in DRDA 2020. Thismay be particularly useful in the context of video content applicationsbecause once the decision is made to view a particular video, a largedata stream may be initiated which can take significant time to displayand can also spend a significant quantity of data against monthlymaximums or otherwise incur additional charges for a user. Also, itmight be useful for a user to be able to quickly and easily obtaininformation about a particular video via RSVP text content withouthaving to leave display of window 2000 so that a user can quickly learnadditional information about several videos before selecting a video forviewing.

Although window 2000 shows a list of featured video items, inalternative embodiments, window 2000 may display a list of video itemsin response to a user search, or it may display items automaticallyrecommended for a particular user.

FIGS. 21a-21g illustrate the playing of a video supplemented by RSVPcontent, the video and associated RSVP content being displayed by anembodiment of the present invention.

In FIG. 21a , shows video display window 2100 displaying an initialframe of the video “My daughter loves walking the dog.” As shown,display window 2100 includes progress bar (timeline bar) 2110. Markers2101, 2102, and 2103 are displayed along progress bar 2110 to indicatetime points in the video that are associated with RSVP content.

When a user selects start icon 2120, the video begins playing normallyin window 2100. When the video reaches frames associated with RSPVcontent, the RSVP content associated with the relevant marker 2101,2102, or 2103 becomes displayable in response to a user selection asshown in subsequent figures.

FIG. 21b illustrates the display in window 2100 at seven seconds intothe video, which, as shown in the progress bar, corresponds to marker2102. At this point in the video, window 2100 also displays marker2102-1 within the video scene to alert the viewer to the presence ofRSVP content associated with that time period in the video. As shown inFIG. 21c , when the user selects marker 2102-1, DRDA 2105 appears andbegins displaying RSVP content. FIGS. 21c-21f show a time sequence asDRDA displays the content associated with display marker 2102-1.Specifically, FIG. 21c shows display at a first time of the word “she.”FIG. 21d shows display at a second time of the word “grabs.” FIG. 21eshows display at a third time of the word “his.” FIG. 21f shows displayat a fourth time of the word “leash.” In this particular embodiment, thewords are displayed such that an optimal recognition position of eachword is at a fixed location within DRDA, the fixed location beingindicated by hash marks 2106. In this particular embodiment, the videoprogress is stopped while the RSVP content is displayed. All the wordsin the RSVP content associated with marker 2102-1 are displayed whilethe video is stopped at a frame corresponding to seven second into thevideo, as shown in FIGS. 21c-21f . However, in alternative embodiments,RSPV content such as that shown in FIGS. 21c-21f could be displayedwithout stopping the video progress. Because RSVP content allows forfaster reading speed than does traditional text presentation, a givenamount of text can be presented via RSPV during a video with less timedisruption to the viewing of the video than if the text were displayedwithout using RSVP.

In the illustrated embodiment, RSPV content is displayed in response toa user interaction. However, an alternative embodiment can be configuredto display embedded RSVP content automatically at particular time pointsin a video.

FIG. 21g illustrates the display in window 2100 at nine seconds into thevideo, which, as shown by progress bar 2110, is past the contentassociated with marker 2102. Because the RSVP content associated with2102 has already been selected (by a selection of marker 2102-1) anddisplayed, marker 2102 is now grayed out to indicate that its associatedcontent has been displayed.

FIGS. 22a-22f illustrate a variation on the example illustrated in FIGS.21a-21g , the illustrated video and RSVP content being displayed by anembodiment of the present invention.

FIG. 22a shows video display window 2100 displaying a frame of the video“My daughter loves walking the dog.” The video frame shown in FIG. 22ais at six seconds into the video. In addition to markers 2101, 2102, and2103 shown on progress bar 2110, window 2100 also displays marker 2104on progress bar 2110 and related marker 2104-1 on the girl who is anidentified object in the video. Marker 2104-1 remains on the identifiedobject while it remains in the frame.

As shown in FIG. 22b , the video has progressed to nine seconds into thevideo and the RSPV content associated with marker 2102 has already beendisplayed (in the manner shown in FIGS. 21c-21f ). Marker 2104-1 remainsdisplayed on the identified object and it is now selected by a userinterface action. As shown in FIG. 22b , when the user selects marker2104-1, DRDA 2105 appears and begins displaying RSVP content. FIGS.22c-21e show a time sequence as DRDA displays the content associatedwith display marker 2104-1. Specifically, FIG. 22b shows display at afirst time of the word “she.” FIG. 22c shows display at a second time ofthe word “calls.” FIG. 22d shows display at a third time of the word“him.” FIG. 22e shows display at a fourth time of the word “Leo.” Inthis particular embodiment, the words are displayed such that an optimalrecognition position of each word is at a fixed location within DRDA,the fixed location being indicated by hash marks 2106. In thisparticular embodiment, the video progress is stopped while the RSVPcontent is displayed. All the words in the RSVP content associated withmarker 2104 are displayed while the video is stopped at a framecorresponding to nine second into the video, as shown in FIGS. 22b-22e .However, in alternative embodiments, RSPV content such as that shown inFIGS. 22b-22e could be displayed without stopping the video progress.

FIG. 22f illustrates the display in window 2100 at thirteen seconds intothe video. In this frame, marker 2104-1 is still shown on the identifiedobject (the girl) and it and related marker 2104 are shown as grayed outto indicate that the RSVP content associated with the object marker hasalready been displayed.

FIGS. 23a-23i show an RSVP-enabled video preview displayed by anembodiment of the present invention. In this embodiment, two excerpts,each comprising frames corresponding to a few seconds, have beenselected to be included in the preview. Also, a few words of text forRSVP display have been selected to be displayed at the beginning of eachexcerpt. FIGS. 23a-23f show RSVP content and video frames correspondingto a first video excerpt and FIGS. 23g-23i show RSVP content and videoframes associate with a second video excerpt.

FIG. 23a shows window 2100 displaying a frame at a beginning of a firstexcerpt, five seconds into the video. FIGS. 23a-23d show a time sequenceas DRDA 2105 displays the content associated with the first excerpt.Specifically, FIG. 23a shows display at a time 1 of the word “she.” FIG.23b shows display at a second time of the word “grabs.” FIG. 23c showsdisplay at a third time of the word “his.” FIG. 23d shows display at afourth time of the word “leash.” FIGS. 23e-23f show two additional videoframes of the first except. In this embodiment, the initial frame of thefirst excerpt (5 seconds into the video) remains displayed while theRSPV content associated with the excerpt is displayed, as shown in FIGS.23a-23d . Once that segment of RSPV content finishes, window 2100displays the remaining frames of the first video excerpt. Two of thoseframes are shown in FIGS. 23e -23 f.

In this embodiment, the RSPV content is displayed at the beginning ofthe excerpt, while the video remains frozen at five seconds into thevideo. However, in alternative embodiments, RSVP content associated withan excerpt including in a video preview may be presented before and/orduring and/or after the playing of the video frames of the excerpt.

FIG. 23g shows window 2100 displaying a frame at a beginning of a secondexcerpt, thirteen seconds into the video. FIGS. 23g-23h show a timesequence as DRDA 2105 displays the content associated with the firstexcerpt. Specifically, FIG. 23g shows display at a first time of theword “bye.” FIG. 23h shows display at a second time of the word “Leo!”FIG. 23i shows an additional video frame of the second except. In thisembodiment, the initial frame of the second excerpt (thirteen secondsinto the video) remains displayed while the RSPV content associated withthe excerpt is displayed, as shown in FIGS. 23g-23h . Once that segmentof RSPV content finishes, window 2100 displays the remaining frames ofthe second video excerpt. One of those frames is shown in FIG. 23 i.

FIGS. 24a-24e illustrate an interface 2400 of a computerized system inaccordance with an embodiment of the present invention for adding RSVPcontent to a video and creating or editing an RSVP-enabled preview ofthe video.

FIG. 24a illustrates an application interface menu 2402 that isactivated within a video editing application by selecting menuactivation icon 2401. Menu 2402 includes option 2403 for inserting anRSVP comment, option 2404 for editing an existing RSVP comment, option2405 for creating or editing an RSPV-enabled preview, and settingsoption 2406.

When a user selects option 2403 from menu 2402 to insert an RSVPcomment, a secondary menu 2403-1 appears. Secondary menu 2403-1 includesoption 2407 for entering a global comment, option 2408 for inserting alocal comment, and option 2409 for inserting an object comment. Ifoption 2407 is selected for inserting a global comment (as shown in FIG.24a ), then a virtual keyboard appears (not separately shown) and theuser can type in a comment that will be associated with the entirevideo. Such a comment will be displayable by RSVP upon, for example,selection of an RSVP marker displayed with a representation of the videoas a video item included in a list of items as illustrated anddescribed, for example, in the context of FIGS. 20a-20d . Alternatively,the user may be directed to speak the comment for entry by a speechrecognition system (not separately shown); type the comment into aphysical keyboard (not separately shown); or provide the comment throughother known input techniques (not separately shown).

FIG. 24b illustrates further aspects of interface 2400 of FIG. 24a . Inparticular, when a user selects option 2408 from menu 2403-1 to insert alocal RSVP comment, then an RSVP marker such as marker 2402-1 appears tothe right of the video window. Instruction 2410 also appears instructingthe user to place the marker at the location within the currentlydisplaying video frame at which the user would like the RSVP marker2402-1 to appear. After the user drags the marker to place it within thevideo frame, a representation 2402 of the marker also appears alongprogress bar 2110. Also, upon placement of the marker within the videoframe, a virtual keyboard appears (not separately shown) and the usercan type in a comment to be displayable by RSVP that will be associatedwith the displayed frame (which corresponds to a time point in thevideo). Alternatively, the user may be directed to speak the comment forentry by a speech recognition system (not separately shown); type thecomment into a physical keyboard (not separately shown); or provide thecomment through other known input techniques (not separately shown). Theentered comment (whether entered by text or by speech) will bedisplayable by RSVP upon, for example, selection of an RSVP markerdisplayed during play back of the video as illustrated and described inthe context of FIGS. 21b -21 f.

FIG. 24c illustrates further aspects of interface 2400 of FIG. 24a . Inparticular, when a user selects option 2409 from menu 2403-1 to insertan RSVP comment to be associated with an object, then object selectionmarkers such as markers 2413, 2414, and 2415 appear to the right of thevideo window. Instruction 2411 also appears instructing the user toplace the markers near different edges of the object with which the RSVPcomment is to be associated. Once the user places markers 2413, 2414,and 2415 within the frame near edges of the selected object (in thisexample, the little girl) then RSVP marker 2419-1 appears on a spot nearthe center of the identified object. In one known video objectrecognition technique, markers on the perimeter of an object can be usedto identify a “hot spot” which tracks the object as it moves insuccessive frames of the video. A related marker 2419 also appears onprogress bar 2110 to indicate the time at which the object-related RSVPcomment will first become selectable by a viewer of the video. Also,once the object is identified and the hot spot marker 2419-1 appears onthe object, a virtual keyboard appears (not separately shown) and theuser can type in a comment to be displayable by RSVP that will beassociated with the identified object. Alternatively, the user may bedirected to speak the comment for entry by a speech recognition system(not separately shown); type the comment into a physical keyboard (notseparately shown); or provide the comment through other known inputtechniques (not separately shown). The entered comment (whether enteredby text or by speech) will be displayable by RSVP upon, for example,selection of an RSVP marker displayed on the object during play back ofthe video as illustrated and described in the context of FIGS. 22a -22f.

FIG. 24d illustrates further aspects of interface 2400 of FIG. 24a .When a user selects option 2404 from menu 2402 to edit an existing RSVPcomment, instruction 2416 then appears asking the user to select themarker associated with the comment that the user would like to edit.This example illustrates the user selecting marker 2403. After the userindicates selection of marker 2403 (for example, by touching it on atouchscreen display), then menu 2404-1 appears. Menu 2404-1 indicatesoption 2421 to edit the RSVP comment or option 2422 to delete thecomment. If option 2422 is selected, a confirmation button appears (notseparately shown) and, if the selection to delete is confirmed, theassociated marker and comment are removed from the video. If option 2421is selected, then a virtual keyboard (not separately shown) and a windowshowing the text of the comment (not separately shown) appear for theuser to make edits to the text of the comment. Alternatively, the usermay be directed to speak edits to the comment for entry by a speechrecognition system (not separately shown); type edits to the commentinto a physical keyboard (not separately shown); or provide edits to thecomment through other known input techniques (not separately shown).

FIG. 24e illustrates further aspects of interface 2400 of FIG. 24a .When a user selects option 2405 from menu 2402 to create or edit an RSVPpreview, then secondary menu 2405-1 appears including options for eachmarker along progress bar 2110 including markers 2401, 2401, and 2403,which are referenced respectively in menu 2405-1 by selectable optionsfor “Marker 1,” “Marker 2,” and “Marker 3.” The interface also displaysthe numerals 1, 2, 3 on those markers along the timeline for the user'sreference. This example shows the user selecting “Marker 3” from menu2405-1. A tertiary menu 2405-2 then appears including option 2417 forsetting the duration of the video excerpt that is to be displayed in thepreview along the RSVP content associated with this marker. Menu 2405-2also includes option 2418 for deleting Marker 3 and its associated RSVPand video content from a video preview created by the user. Uponselection of option 2417, a user interface (not separately shown)appears allowing the user to enter a video excerpt duration in secondsor in frames that will include video frames including and immediatelyfollowing the time point of the marker. The interface may be implementedthrough a virtual keyboard, speech recognition feature, or otherhuman-to-computer interface.

FIGS. 25a-25k illustrate an augmented reality (AR) applicationsupplemented by RSVP content in accordance with an embodiment of thepresent invention. AR is well known and typically involves viewing realworld objects through an AR viewer which provides information aboutthose objects. One example of a device in which an AR viewer may beimplemented is a camera view finder associated with a smart phone cameraapplication. Another example is through a head mounted or other near eyedisplay. Examples of such displays and associated RSVP implementationsare illustrated and described further in the context of FIGS. 26a-26cand 27a-27d . In some implementations, the user may view the real worldeither directly (unmediated by any optical material) or through anoptical material, and then view relevant information about the realworld objects in a near-eye display placed near or in a portion of theuser's field of vision. Such implementations are compatible withembodiments of the present invention. However, the implementationillustrated in FIGS. 25a-25k assumes that the user is viewing the realworld through an AR application window implemented in conjunction with acamera view finder of a smartphone.

FIG. 25a illustrates an AR application window 2500 prior to activationof AR-related display objects. As shown, a user is viewing a real-worldurban scene (in New York City) through the screen view of a smart phonecamera application (which typically is able to display a real-worldscene on the smart phone display screen).

FIG. 25b illustrates the same scene after the associated AR applicationhas recognized particular real world objects. As shown, window 2500 nowdisplays markers 2501 and 2502 to indicate the presence of RSVP contentassociated with particular real world objects. Specifically, marker 2501indicates the presence of RSVP content associated with the statue of abull and marker 2502 indicates the presence of RSVP content associatedwith the subway station entrance located by the bull and visible justbeneath the bull's left ear.

FIGS. 25c-25f show a time sequence of the display of RSVP contentassociated with the statue after a user-interface action indicatesselection of marker 2501. Upon selection of the marker, DDRA 2503appears and begins displaying words. In this embodiment, DDRA displaysthe words such that an optimal recognition character position is place afixed location of the display indicated by hash marks 2504. FIG. 25cshows display of the word “Charging” at a first time. FIG. 25d showsdisplay of the word “Bull” at a second time. FIG. 25e shows display ofthe word “by” at a third time. FIG. 25f shows display of the word“DiModica” at a fourth time. Window 2500 shows marker 2501 as grayed outin FIGS. 25c-25f to indicate that display of the RSVP content associatedwith the marker has been initiated. In the illustrated embodiment, themarker is grayed out as soon as the associated RSVP display begins. Inalternative embodiments, the marker can be grayed out when or after theRSVP display is finished.

FIG. 25g shows window 2500 after the RSVP content associated with marker2501 has finished displaying. In this embodiment, the DDRA is no longerdisplayed when RSVP content is either not selected or has finisheddisplaying. This minimizes obstruction of the user's view of real worldobjects when RSPV content is not actively displaying. However, inalternative embodiments, a DDRA such as DDRA 2503 may remain on thedisplay view when RSVP content is temporarily not displayed.

FIGS. 25h-25k show a time sequence of the display of RSVP contentassociated with the subway entrance after a user-interface actionindicates selection of marker 2502. Upon selection of the marker, DDRA2503 appears again and begins displaying words. FIG. 25h shows displayof the word “Bowling” at a first time. FIG. 25i shows display of theword “Green” at a second time. FIG. 25j shows display of the word“Subway” at a third time. FIG. 25k shows display of the word “100 feet”at a fourth time. Window 2500 shows marker 2502 as grayed out in FIGS.25h-25k to indicate that display of the RSVP content associated with themarker has been initiated.

FIGS. 26a-26c show various types of alternative known display devices inwhich embodiments of the invention might be implemented. Whileparticular embodiments of the invention can be implemented in thecontext of any electronic device capable of displaying information,small screen display devices such as smart phones and other devicesderive particular benefits from the ability of invention embodiments toincrease the amount and speed of information displayable within alimited screen space. Other embodiments of the invention have alreadybeen illustrated and described in the context of smart phone displays.However, FIGS. 26a-26c show alternative displays devices which also havea particular need for the benefits provided by implementation ofembodiments of the present invention. FIG. 26a illustrates a headmounted display HMD device 2600 a implemented in the form factor oftraditional eyeglasses. In particular, device 2600 a includes display2601 a. FIG. 26b illustrates another HMD device 2600 b, which includesdisplay 2601 b. FIG. 26c illustrates a smart watch device 2600 cincluding a display 2601 c.

FIGS. 27a-27d show a time sequence of display of RSVP content in anRSVP-enabled interface implemented in an HMD device such as, forexample, one of the HMD devices shown in FIGS. 26a-26b . Window 2700shows the user's field of vision from inside of a shopping mall. TheRSVP content is displayed within DRDA 2701 on display 2702. In thisexample, the user is receiving a message from a friend who is also atthe mall. FIG. 27a shows display of the word “What” at a first time.FIG. 27b shows display of the word “store” at a second time. FIG. 27cshows display of the word “you” at a third time. FIG. 27d shows displayof the word “at?” at a fourth time.

FIGS. 28a-28b show a process flow for a method 2800 implemented by acomputer system in accordance with an embodiment of the presentinvention for displaying RSVP content within a video player application.Method 2800 may be implemented, for example, on a user device executinga video player application such as application 210 a of FIG. 2b that isenabled to process RSPV content by RSVP software 211 a which utilizesRSVP library 204.

Referring to FIG. 28a , step 2801 opens an RSVP-enabled video file inresponse to user input 2819. Step 2802 determines whether the filecontains any RSVP content. If no, then step 2803 proceeds to play thevideo in accordance with the video application's normal (i.e., non-RSPV)video playing execution. If yes, the step 2841 displays one or more RSVPmarkers along the video progress bar/time line. Step 2804 determineswhether the user has chosen (based on user input 2820) to have RSPVcontent in the video file be played automatically. If no, the methodproceeds to step 2823 of FIG. 28b (described further below). If yes,then step 2805 begins playing the video content. In some embodiments,user input 2820 may be provided after each RSVP-enabled video file isopened. In other embodiments, user input 2820 may be provided through auser settings interface for determining default user preferences withrespect to RSVP-enabled video playback (settings interface notseparately shown).

During video playback by step 2805, step 2809 determines whether thenext RSVP location in the video file has been reached. If no, then step2807 determines whether the video has finished. If yes, then videoplayback ends at step 2806. If no, then step 2805 continues playing thevideo. However, if the result of step 2809 is yes, then step 2810 pausesvideo playback. Step 2813 displays the associated RSVP content in aDRDA.

Step 2814 determines whether there is RSVP sub-content associated withthe RSVP content displayed in step 2813. If no, then the method proceedsto step 2807. If yes, then step 2815 display titles for the sub-content(see, e.g., the example shown in FIG. 7d ). In this embodiment, thetitles of sub-content are shown for 2 seconds. However, alternatives mayshow the titles for different amounts of time. Step 2818 determineswhether user input 2821 has indicated that the user has selected to viewthe sub-content. If no, then step 2817 changes the marker correspondingto the primary RSVP content to indicate that it has been read (e.g., bygraying it out as illustrated in previous figures). If yes, then step2816 displays the sub-content in the DRDA (see, e.g., as shown in FIGS.7e-7h ). When the sub-content is finished displaying, then step 2817changes the RSVP marker as previously described and the method returnsto step 2804 to continue playing the video while step 2809 determineswhether the next RSVP content segment has be reached during videoplayback.

Referring to FIG. 28b , if RSVP content has not been designated todisplay automatically in step 2804 (FIG. 28a ), then step 2823 proceedsto play the video in accordance with the video application's normalvideo playing execution. During video playback, step 2824 determineswhether the next RSVP location in the video file has been reached. Ifno, then step 2826 determines whether the video has finished. If yes,then video playback ends at step 2837. If no, then step 2823 continuesplaying the video. However, if the result of step 2824 is yes, then step2838 determines whether the RSVP content at the current video locationis object based (e.g., as shown and described in the context of FIGS.22a-22f ). If yes, then step 2841 shows an RSPV marker on the relevantobject in the video frame. If no, then step 2825 shows an RSVP marker inthe frame in a designated location (see, e.g., marker 2102-1 in FIG. 21b; in some embodiments, the location may have been designated by an RSVPcontent author as illustrated in FIG. 24b with respect to marker2402-1).

Step 2827 determines, based on user input 2836, whether the user hasselected to view the RSVP content associated with any of the marker(s)currently displayed in the video frame. If no, then the method returnsto step 2823. If yes, then step 2829 pauses video playing and step 2830displays the corresponding textual content in a DRDA using RSVP.

Step 2830 determines whether there is RSVP sub-content associated withthe RSVP content displayed in step 2831. If no, then the method proceedsto step 2826. If yes, then step 2832 display titles for the sub-content.Step 2833 determines whether user input 2835 has indicated that the userhas selected to view the sub-content. If no, then step 2834 changes themarker corresponding to the primary RSVP content to indicate that it hasbeen read (e.g., by graying it out as illustrated in previous figures).If yes, then step 2828 displays the sub-content in the DRDA. When thesub-content is finished displaying, then step 2834 changes the RSVPmarker as previously described and the method returns to step 2823 tocontinue playing the video while step 2824 determines whether the nextRSVP content segment has been reached during video playback.

FIGS. 29a-29b show a process flow for a method 2900 implemented by acomputer system in accordance with an embodiment of the presentinvention enabling a user to add RSVP content to a video. Such a previewmay be created, for example, through a user interface as shown anddescribed FIGS. 24a-24e . However, in this and other examplesillustrated herein of various embodiments of aspects of the invention,the exact logic shown in flow diagrams is not necessarily intended tocorrespond to the exact interface illustrated in other figures. Both aremerely intended to provide examples and those skilled in the art will ofcourse appreciate that there are many variations that would not at alldepart from the spirit and scope of the present invention.

Method 2900 may be implemented, for example, on a user device executinga video editing/annotating application such as application 210 b of FIG.2b that is enabled to process RSPV content by RSVP software 211 b whichutilizes RSVP library 204.

Referring to FIG. 29a , step 2901 opens the video to which RSVP contentis to be added. Step 2902 determines, based on user input 2935, whetherthe user has selected to add or edit RSPV content related objects in thevideo. If yes, then the method proceeds to step 2922, shown anddescribed below in the context of FIG. 29b . If no, then the methodproceeds to step 2905. Step 2905 determines, based on user input 2917,whether the user has selected to add or edit a global RSVP comment. Ifyes, then step 2904 displays an interface for the user to provide input2918 to add or edit a global comment to the video that will bedisplayable by RSVP. The method then proceeds to step 2906. If theresult of step 2905 is no, then the method proceeds to step 2906directly.

Step 2906 determines, based on user input 2916, whether RSVP content isto be added to the video timeline to make the content displayable as aparticular time in the video. If no, the step 2907 saves the video andexits. If yes, the step 2908 advances the video, based on user input2921 (e.g., selecting a point on a progress bar), to the time locationin the video where the RSVP content is to be added. Step 2909 thendisplays an interface to the user to add content to be displayed viaRSPV. As described previously in the context of FIGS. 24a-24d , such aninterface may include a virtual keyboard, speech recognition interface,or other user interfaces for entering or editing textual content. Step2913 receives user input 2919 that enters or edits textual content thatis to be displayable to a content consumer via RSVP. Step 2910 thendetermines whether the user would like to add any secondary RSVPcontent. As previously described in the context of FIGS. 7d-7h ,indicators for such additional content may be provided at the end of aprimary RSVP display so that a content consumer can determine whether toselect to view the additional content. The concepts illustrated therealso apply to RSVP content added to video.

If the result of step 2910 is yes, then step 2911 displays an interfaceto receive user input 2920 to add secondary content to be displayable byRSPV. The method then proceeds to step 2903 to receive the enteredcontent. The method then proceeds to step 2912. If the result of step2910 is no, then the method proceeds directly to step 2912. Step 2912,based on user input 2915, selects how many frames should be associatedwith the currently entered RSPV content (including the primary RSPVcontent received at step 2913 and any secondary content received at step2914). In various alternatives, the frames may be represented by thenumber of frames or in a time duration of associated video content. Themethod then returns to step 2908.

FIG. 29b shows the flow of a portion of method 2900 that is executed ifthe result of step 2902 (in FIG. 29a ) is yes and user added content isto be associated with objects. Step 2922 advances the video based onuser input 2929 to the time location in the video where the RSVP contentis to be added. Step 2923 provides an interface and accepts user inputto identify a hotspot on a video object selected by the user forassociation with RSPV content. Step 2933 then displays an interface tothe user to add content to be displayed via RSPV. Step 2924 receivesuser input 2931 that enters or edits textual content that is to bedisplayable to a content consumer via RSVP. Step 2925 then determineswhether the user would like to add any secondary RSVP content.

If the result of step 2925 is yes, then step 2926 displays an interfaceto receive user input 2932 to add secondary content to be displayable byRSPV. The method then proceeds to step 2927 to save the RSVP enteredcontent and any secondary entered RSPV content along with the identifiedobject hotspot and to place an associated marker on the video timelinebar. If the result of step 2925 is no, then the method proceeds directlyto step 2927. The method then proceeds to step 2928 to determine, basedon user input 2937, whether the user would like to add any more RSPVcontent to objects in the video. If the result of step 2928 is yes, thenthe method returns to step 2922. If the result of step 2928 is no, thenstep 2934 saves the video and the process ends.

FIG. 30 illustrates a process flow for a method 3000 implemented by acomputer system in accordance with an embodiment of the presentinvention for playing an RSVP-enabled video preview. Such a preview maybe displayed, for example, as shown and described FIGS. 23a-23g . Method3000 may be implemented, for example, on a user device executing a videoplayer application such as application 210 a of FIG. 2b that is enabledto process RSPV content by RSVP software 211 a which utilizes RSVPlibrary 204.

Step 3001 opens the video preview (which may be a separate file or datastream distinct from the underlying video that is the subject of thepreview while at the same time typically containing a portion of thesame of the same content included in the underlying video). Step 3002determines if the preview has any RSVP content. If no, then step 3003displays the video preview normally (i.e., as it would any other videowithout RSVP content). If yes, then step 3004 displays RSVP markers onthe video timeline (progress bar). Step 3007 then moves the currentlocation indication on the progress bar to the first RSVP markerlocation. Step 3010 displays the RSVP content associated with the markerin a DRDA. After the first RSVP content segment finishes displaying,then step 3009 plays the video excerpt that the preview file hasassociated with that RSVP segment. Step 3008 then changes the currentRSVP marker to indicate that the RSVP content associated with thatmarker has been displayed (e.g., by graying out the marker). Step 3005determines whether the video preview contains additional RSVP-enabledcontent. If no, then step 3006 stops the video playback and displays thevideo timeline with markers. In one alternative, the preview markers areindividually selectable so that the user can go back and view individualsegments without having to view the entire preview. If the result ofstep 3005 is yes, then step 3007 repeats and advances to the next RSVPmarker location in the timeline.

FIG. 31 illustrates a process flow for a method 3100 implemented by acomputer system in accordance with an embodiment of the presentinvention for delivering augmented reality (“AR”) content elements thatare enabled to be displayed via RSPV. Such a method may be carried outby, for example, an AR service application such as application 217 shownin FIG. 2d including RSVP server library 218 and executing on a server101.

Step 3101 creates an AR element in the relevant AR service application.Such elements are well known and typically include displayableinformation that is associated with a real world object viewable by auser of the AR service. Step 3102 determines whether the AR element isdesignated to be displayable using RSVP. If no, then the processproceeds to step 3104. If yes, then step 3103 prepares the text for RSPVdisplay. In particular embodiments, this may include various additionalsteps such as, for example, parsing it into a plurality of displayableelements, inserting blank elements where appropriate, calculatingdisplay time multipliers based on word length, and determining anoptimal recognition position character or proportionate position asfurther described in co-pending U.S. application Ser. No. 13/547,982incorporated herein by reference above. As is further explained therein,in a particular implementation, some or all steps for preparing text forRSVP display may be accomplished on a user device rather than remotelyon a server computer. If there is any sub-content associated with the ARelement, then step 3103 also prepares that sub-content for RSVP display.

Step 3104 saves the AR element on the server. Step 3105 receives, at theAR service application, a request for a particular AR element from auser device. Step 3106 transmits the AR element to the requesting userdevice.

FIG. 32 illustrates a process flow for a method 3200 implemented by acomputer system in accordance with an embodiment of the presentinvention for processing and displaying AR content elements that areenabled to be displayed via RSPV. Such a method may be carried out by,for example, an AR application such as application 212 shown in FIG. 2cincluding RSVP software 213 and executing on a user device 210. Thoseskilled in the art will appreciate that, in a typical implementation, anAR application such as application 212 and/or AR service such as ARservice 217 will rely in part on information obtained from a locationsensor such as GPS 212 of FIG. 2c . An example of one possible resultinguser interface for an RSVP-enabled AR application implementing a methodaccording to an aspect of an invention embodiment is illustrated anddescribed in the context of FIGS. 25a -25 k.

Process 3200 begins with step 3207 which detects AR elements receivedfrom an AR service (such as AR service 217 executing method 3100 to sendAR elements including RSVP displayable content to a user device). Step3208 determines whether the received AR element or elements include RSVPcontent. If no, then step 3209 proceeds as the AR application normallywould without RSPV content. If yes, the step 3210 displays RSVP markerson objects views associated with real world objects. In one embodiment,these markers include icons such as those illustrated in FIGS. 25a-25k .In an alternative embodiment, these markers may instead be or includeshort textual titles identifying the real world object. Step 3211receives user input 3217 selecting a particular RSVP marker for display.Step 3216 displays the RSVP content corresponding to the selected markerin a DRDA. Step 3215 then determines whether there is any sub RSVPcontent for display. If no, then step 3219 changes the markercorresponding to the RSVP content to indicated that it has been read. Ifyes, then step 3214 displays titles for the sub-content for two seconds.Step 3213 determines whether user input 3218 has indicated that the userhas selected to view the sub-content. If no, then step 3219 changes themarker corresponding to the primary RSVP content to indicate that it hasbeen read. If yes, then step 3216 displays the sub-content in the DRDA.When the sub-content is finished displaying, then the process returns tostep 3215 to determine if there is any additional sub-content(sub-content can also have additional sub-content). Once all the RSVPcontent and sub-content associated with a particular AR element haseither been displayed or declined by the user for display, the processthen returns to step 3211 to wait for selection of another RSPV markerfor RSVP display of content associated with another AR element.

While the present invention has been particularly described with respectto the illustrated embodiments, it will be appreciated that variousalterations, modifications and adaptations may be made based on thepresent disclosure, and are intended to be within the scope of thepresent invention. While the invention has been described in connectionwith what are presently considered to be the most practical andpreferred embodiments, it is to be understood that the present inventionis not limited to the disclosed embodiments but, on the contrary, isintended to cover various modifications and equivalent arrangementsincluded within the scope of the appended claims.

What is claimed is:
 1. A method of providing for display of informationregarding video content on a user electronic device, the user devicebeing enabled to display an interface comprising a plurality ofselectable video item representations, the method comprising: enablingthe user device to display, in response to a user interface actionselecting to receive additional information regarding one of a pluralityof video items corresponding to a selected one of the plurality of videoitem representations, a designated display area while simultaneouslycontinuing to display at least a portion of the interface comprising theplurality of video items, wherein continuing to display at least aportion of said interface comprises displaying unselected others of theplurality of selectable video item representations, the designateddisplay area being designated for rapid serial visual presentation(RSVP); and enabling the user device to display in the designateddisplay area, in response to the user interface action, textual contentcontaining the additional information using RSVP, wherein display of thetextual content using RSVP comprises sequentially displaying respectivedisplay elements comprising respective portions of the textual contentsuch that a predetermined position of a respective display element isdisplayed at a fixed display location, and further wherein eachrespective portion of the textual segment fits within a perceptual spanof 20 characters or less, each respective portion including a subset ofthe textual content.
 2. The method of claim 1 wherein the textualcontent corresponding to the selected video item is displayable withoutdownloading a video file associated with the selected video item.
 3. Themethod of claim 1 wherein the textual content corresponding to theselected video item is displayable without receiving a stream of videocontent associated with the selected video item.
 4. The method of claim1 wherein the plurality of selectable video item representations arereceived from a server computer in response to a search query sent fromthe user device.
 5. The method claim 1 wherein the plurality ofselectable video item representations are received from a servercomputer in response to the user device accessing a listing of featuredvideo items.
 6. The method of claim 1 wherein a plurality of seriallydisplayed words in the textual content are displayed such that anoptimal recognition position of each of the plurality of seriallydisplayed words is placed at a fixed location on a display of the userelectronic device.
 7. The method of claim 1 wherein the textual contentincludes primary content and sub-content, the sub-content beingselectable by a user interface action during a time period after RSVPdisplay of the primary content is initiated, selection of thesub-content initiating RSVP display of the sub-content.
 8. The method ofclaim 7 wherein the time period occurs after RSVP display of the primarycontent has finished.
 9. The method of claim 7 wherein a selectabletitle of the sub-content is displayed for selection of the sub-contentby the user interface action.
 10. A method of providing fortext-enhanced video display on a user electronic device, the methodcomprising: enabling the user electronic device to receive media contentthat includes video content and related textual content, the relatedtextual content being received for display by rapid serial visualpresentation (RSVP) on the user electronic device; and enabling the userelectronic device to, with display of the video content, display therelated textual content in a designated display area on the user deviceusing RSVP corresponding to a marker displayed at a particular positionin one or more frames of the video content, the particular positionbeing associated with a particular object within the video content. 11.The method of claim 10 wherein the related textual content is displayedautomatically at the one or more time locations within the videocontent.
 12. The method of claim 10 wherein playing of the video contentis paused while the related textual content is displayed using RSVP. 13.The method of claim 10 wherein playing of the video content continueswhile the related textual content is displayed using RSVP.
 14. Themethod of claim 1 wherein one or more markers that indicate presence oftextual content displayable by RSVP are displayed along a progress barinterface that indicates progress of playing of the video content. 15.The method of claim 10 wherein the textual content includes primarycontent and sub-content, the sub-content being selectable by a userinterface action during a time period after RSVP display of the primarycontent has begun, selection of the sub-content by the user interfaceaction being operable to initiate RSVP display of the sub-content. 16.The method of claim 15 wherein the time period occurs after RSVP displayof the primary content has finished.
 17. The method of claim 15 whereina selectable title of the sub-content is displayed for selection of thesub-content by the user interface action.
 18. A method of providing fortext-enhanced video display on a user electronic device, the methodcomprising: enabling the user electronic device to receive media contentthat includes video content and related textual content, the relatedtextual content being received for display by rapid serial visualpresentation (RSVP) on the user electronic device, the video contentcomprising one or more respective video excerpts providing partial videocontent of video item, the related textual content comprising one ormore respective textual content segments corresponding to the one ormore respective video excerpts; and enabling the user electronic deviceto display a first textual content segment using RSVP and display afirst video excerpt with the first textual content segment, whereindisplaying a first textual content segment using RSVP comprisessequentially displaying respective display elements comprisingrespective portions of the first textual segment such that apredetermined position of a respective display element is displayed at afixed display location, and further wherein each respective portion ofthe first textual segment fits within a perceptual span of 20 charactersor less, each respective portion including a subset of the first textualsegment.
 19. The method claim 18 wherein enabling the user electronicdevice to display the first video excerpt with the first textual contentsegment comprises enabling the user electronic device to display thefirst textual segment prior to displaying the first video excerpt. 20.The method of claim 19 wherein the first video excerpt displays inresponse to a user interface action selecting the first video excerpt,the first user interface action being receivable when or after the firsttextual content segment is displayed using RSVP.
 21. The method of claim1 wherein the user electronic device is a smart phone.
 22. The method ofclaim 1 wherein the user electronic device is a smart watch.
 23. Themethod of claim 1 wherein the user electronic device is a head mounteddisplay.
 24. The method of claim 10 wherein the user electronic deviceis a smart phone.
 25. The method of claim 10 wherein the user electronicdevice is a smart watch.
 26. The method of claim 10 wherein the userelectronic device is a head mounted display.
 27. The method of claim 18wherein the user electronic device is a smart phone.
 28. The method ofclaim 18 wherein the user electronic device is a smart watch.
 29. Themethod of claim 18 wherein the user electronic device is a head mounteddisplay.